Recognition apparatus, method, and computer program product

ABSTRACT

In an embodiment, a recognition apparatus includes: an obtaining unit configured to obtain positions of a specific part in a coordinate system having a first axis to an n-th axis (n≧2); a calculating unit configured to calculate a movement vector of the specific part; a principal axis selecting unit configured to select a principal axis; a turning point setting unit configured to set a position at which there is a change in the principal axis, and set a position at which there is a change; a section setting unit configured to set a determination target section, and set a previous section; a determining unit configured to calculate an evaluation value of the determination target section and an evaluation value of the immediately previous section and, determine which of the first axis to the n-th axis is advantageous; and a presenting unit configured to perform the determined result.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2011-167608, filed on Jul. 29, 2011; theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to a recognitionapparatus, a recognition method, and a computer program product.

BACKGROUND

Regarding recognition apparatuses configured to recognize gestures, atechnology is known in which, depending on the angle of a velocityvector calculated in a section in which the velocity of a specific part,or a specific bodily part, of a photographic subject is equal to orgreater than a threshold value, it is determined whether the specificpart performs a gesture (movement) in the vertical direction or in thehorizontal direction.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a configuration diagram illustrating an example of arecognition apparatus according to a first embodiment;

FIG. 2 is a diagram illustrating an example of a gesture of a specificpart according to the first embodiment;

FIG. 3 is a diagram illustrating an example of positions of the specificpart;

FIG. 4 is a diagram illustrating an example of movement vectorsaccording to the first embodiment;

FIG. 5 is an explanatory diagram for explaining an example of settingdetermination target sections and respective previous sections accordingto the first embodiment;

FIG. 6 is a diagram illustrating another example of a gesture performedby the specific part according to the first embodiment;

FIG. 7 is a diagram illustrating an example of presentation according tothe first embodiment;

FIG. 8 is a flowchart for explaining an exemplary sequence of operationsperformed by the recognition apparatus according to the firstembodiment;

FIG. 9 is a flowchart for explaining an exemplary sequence of operationsperformed by a turning point setting unit in the recognition apparatusaccording to the first embodiment;

FIG. 10 is a flowchart for explaining an exemplary sequence ofoperations performed by a determining unit in the recognition apparatusaccording to the first embodiment;

FIG. 11 is a configuration diagram illustrating an example of arecognition apparatus according to a second embodiment;

FIG. 12 is a diagram illustrating another example of a gesture performedby the specific part according to the second embodiment;

FIG. 13 is a diagram for illustrating an example of axes each beingadvantageous at a particular progress according to the secondembodiment;

FIG. 14 is a diagram for illustrating another example of axes each beingadvantageous at a particular progress according to the secondembodiment;

FIG. 15 is an example of supplementary presentation according to thesecond embodiment;

FIG. 16 is a diagram illustrating another example of supplementarypresentation according to the second embodiment;

FIG. 17 is a diagram illustrating still another example of supplementarypresentation according to the second embodiment;

FIG. 18 is a flowchart for explaining an exemplary sequence ofoperations performed in the recognition apparatus according to thesecond embodiment;

FIG. 19 is a flowchart for explaining an exemplary sequence ofoperations performed by a determining unit in the recognition apparatusaccording to the second embodiment;

FIG. 20 is a flowchart for explaining another exemplary sequence ofoperations performed by the determining unit in the recognitionapparatus according to the second embodiment;

FIG. 21 is a configuration diagram illustrating an example of arecognition apparatus according to a third embodiment;

FIG. 22 is a diagram of an example of setting second-axis directionturning points according to the third embodiment;

FIG. 23 is a diagram illustrating exemplary advantageous directions eachcorresponding to a progress rate according to the third embodiment;

FIG. 24 is a diagram of feed action candidates each selectedcorresponding to a progress rate according to the third embodiment;

FIG. 25 is a flowchart illustrating an exemplary sequence of operationsperformed in the recognition apparatus according to the thirdembodiment;

FIG. 26 is a flowchart for explaining an exemplary sequence ofoperations performed by a second axis determining unit in therecognition apparatus according to the third embodiment;

FIG. 27 is a diagram illustrating an exemplary coordinate systemaccording to a fourth embodiment;

FIG. 28 is a diagram illustrating another exemplary coordinate systemaccording to the fourth embodiment;

FIG. 29 is a diagram illustrating a television receiver according to afirst application;

FIG. 30 is a diagram illustrating a personal computer according to asecond application; and

FIG. 31 is a diagram illustrating a medical information terminalaccording to a third application.

DETAILED DESCRIPTION

According to an embodiment, a recognition apparatus includes: anobtaining unit; a calculating unit; a principal axis selecting unit; aturning point setting unit; a section setting unit; a determining unit;and a presenting unit. The obtaining unit is configured to obtain, inchronological order, positions of a specific part of a photographicsubject that are represented in a coordinate system having a first axisto an n-th axis (n≧2). The calculating unit is configured to calculate amovement vector of the specific part at each of the positions obtainedin chronological order. The principal axis selecting unit is configuredto calculate, for each of the movement vectors that has been calculated,a first inner product that is an inner product of the movement vectorand the first axis to an n-th inner product that is an inner product ofthe movement vector and the n-th axis, and select, as a principal axis,an axis that is used in calculating the inner product, from among thefirst inner product to the n-th inner product, having the greatestmodulus. The turning point setting unit is configured to set a positionat which there is a change in the principal axis as a principal axisturning point and a direction turning point, and set a position at whichthere is a change in the direction of any of the movement vectors in theprincipal axis component as the direction turning point. The sectionsetting unit is configured to set a determination target section, inwhich a start point is either set to the latest principal axis turningpoint when the principal axis turning points are present or set to aninitial point indicating the initially-obtained position when theprincipal axis turning points are absent, and in which an end point isset to a latest point that indicates the most-recently obtainedposition, set a previous section, in which an end point is set to thestart point of the determination target section and in which a startpoint is either set to a direction turning point present immediatelybefore the end point when the direction turning points are presentbefore the end point or set to the initial point when the directionturning points are not present immediately before the end point. Thedetermining unit is configured to calculate an evaluation value of thedetermination target section and an evaluation value of the immediatelyprevious section and, based on the evaluation value of the determinationtarget section and the evaluation value of the previous section,determine which of the first axis to the n-th axis is advantageous formovement of the specific part in the determination target section. Thepresenting unit is configured to perform presentation according to adetermination result obtained by the determining unit.

First Embodiment

FIG. 1 is a configuration diagram illustrating an example of arecognition apparatus 100 according to a first embodiment. Asillustrated in FIG. 1, the recognition apparatus 100 includes anobtaining unit 102, a calculating unit 104, a principal axis selectingunit 106, a turning point setting unit 108, a section setting unit 110,a determining unit 112, and a presenting unit 114.

The obtaining unit 102 obtains, in chronological order, the positions ofa specific part, or a specific bodily part of a photographic subjectthat are represented in a coordinate system having a first axis and asecond axis. For example, with respect to chronological images capturedusing an imaging element such as an image sensor, the obtaining unit 102detects the position of the specific part from each image using at leastone of the techniques such as the pattern recognizing technique, thebackground differencing technique, the skin color extracting technique,and the inter-frame differencing technique; and obtains the detectedpositions of the specific part. Herein, as a position of the specificpart, it is possible to consider the center of gravity of the detectedarea of the specific part.

FIG. 2 is a diagram illustrating an example of a gesture of a specificpart according to the first embodiment. FIG. 3 is a diagram illustratingan example of positions of the specific part that are obtained inchronological order by the obtaining unit 102. For example, when thespecific part follows a movement trajectory as illustrated in FIG. 2;the obtaining unit 102 obtains discreet positions P₁ to P₆, which aresampled from time-series images, as the positions in chronological orderof the specific part as illustrated in FIG. 3.

In the first embodiment, the explanation is confined to an example inwhich the specific part points to a hand of the photographic subject(user). However, alternatively, the specific part can also be thefingers of the photographic subject, the head of the photographicsubject, or a light emitting unit of a laser point.

Herein, the positions of the specific part are represented in thecoordinate system having the first axis and the second axis. Thus, aslong as the first axis and the second axis are discrimination targetsfor the gestures (movement) of the specific part, they serve thepurpose. In the first embodiment, it is an object to discriminate thegestures in the vertical (up-down) direction from the gestures in thehorizontal (left-right) direction. With that in mind, the first axis istreated as the axis in the vertical direction and the second axis istreated as the axis in the horizontal direction, and it is assumed thatthe first axis and the second axis intersect with each other. However,that is not the only possible case. For example, if the object is todiscriminate the gestures in the northwest-southeast direction from thegestures in the northeast-southwest direction, then the first axis canbe treated as the axis in the northwest-southeast direction and thesecond axis can be treated as the axis in the northeast-southwestdirection. Still alternatively, depth information that is estimated froman imaging element or from a distance sensor also can be considered aseither one of the first axis and the second axis.

Moreover, the positions of the specific part that are expressed in thecoordinate system having the first axis and the second axis can eitherbe expressed by the coordinates having those axes as the components orbe expressed by the coordinates defined on images captured by theimaging element (expressed by pixel positions). Furthermore, thecoordinates expressing the positions of the specific part can either beabsolute coordinates or be relative coordinates from the position of theface of the user. Moreover, the positions of the specific part can becoordinate values normalized from the face of the user or from theinitially-detected size of the specific part.

Meanwhile, the obtaining unit 102 can be configured not only to obtainthe positions of the specific part but also to obtain the timings ofobtaining the positions. For example, when the positions of the specificpart are expressed in x and y coordinates on images; then, in an i-thframe, the obtaining unit 102 obtains a position (x_(i), y_(i)) as wellas a timing T_(i). The timing can be expressed as, for example, theelapsed time, the elapsed clock count, or the elapsed frame count basedon the activation time of the recognition apparatus 100 or based on thestart time of obtaining positions of the obtaining unit 102. Herein,although the explanation is given for a case of obtaining the positionfor each obtained frame, it is not the only possible case.

The calculating unit 104 calculates a movement vector of the specificpart at each position obtained in chronological order. In the firstembodiment, the explanation is given for a case when each movementvector is a relative vector with respect to the previously-obtainedposition obtained by the obtaining unit 102. However, that is not theonly possible case.

FIG. 4 is a diagram illustrating an example of movement vectorscalculated by the calculating unit 104 according to the firstembodiment. For example, when the obtaining unit 102 obtains theposition P₂, the calculating unit 104 calculates a movement vector R₁that is the relative vector obtained with respect to thepreviously-obtained position P₁ (i.e. the position P₁ obtained in theprevious frame). More particularly, if the position P₁ has coordinates(x₁, y₁) and the position P₂ has coordinates (x₂, y₂) (herein, thesecond axis is assumed to be the x axis and the first axis is assumed tobe the y axis); then the calculating unit 104 calculates a movementvector R₁=(x₂−x₁, y₂−y₁)^(T), where T represents a transposed matrix.

Meanwhile, a movement vector need not be a relative vector with respectto the previously-obtained position. Alternatively, a movement vectorcan be a relative vector with respect to a position obtained at somenumber of times earlier (obtained at some number of frames earlier) orcan be a relative vector with respect to a direction turning point(described later in detail) present at the previous position. Moreover,a movement vector can be vector that has a slope calculated from thepreviously-obtained position (i.e., the position obtained in theprevious frame), the currently-obtained position (i.e., the positionobtained in the current position), and the subsequently-obtainedposition (i.e., the position obtained in the subsequent frame); and thatpasses through the currently-obtained position. If thepreviously-obtained position P₁ has coordinates (x₁, y₁), thecurrently-obtained position P₂ has coordinates (x₂, y₂), and thesubsequently-obtained position P₃ has coordinates (x₃, y₃); then, forexample, the abovementioned slope can be considered to be the average ofa slope (y₂−y₁/x₂−x₁) of the vector between the positions P₁ and P₂ anda slope (y₃−y₂/x₃−x₂) of the vector between the positions P₂ and P₃.Alternatively, a movement vector can be a vector that has a slopecalculated by estimating the movement direction of the specific partfrom the direction of the blur of the specific part on images capturedby an imaging element; and that passes through the currently-obtainedposition.

For each movement vector that is calculated, the principal axisselecting unit 106 calculates a first inner product that is the innerproduct of the movement vector and the first axis and calculates asecond inner product that is the inner product of the movement vectorand the second axis. Then, of the first axis and the second axis, theprincipal axis selecting unit 106 selects the axis that is used incalculating the inner product, from among the first inner product andthe second inner product, having the greater modulus as the principalaxis. That is, of the first axis and the second axis, the principal axisselecting unit 106 selects the axis closer to the direction of gesturesof the specific part.

For example, as illustrated in the example in FIG. 4, the first axis andthe second axis intersect at a right angle. Thus, if the slope of amovement vector with respect to the first axis is within the range of±45°, that is, if the acute angle between the first axis and a movementvector is within the range of ±45°; then the principal axis selectingunit 106 selects the first axis as the principal axis. On the otherhand, if the slope of a movement vector with respect to the second axisis within the range of ±45°, that is, if the acute angle between thesecond axis and a movement vector is within the range of ±45°; then theprincipal axis selecting unit 106 selects the second axis as theprincipal axis. As a result, the principal axis selecting unit 106selects the first axis as the principal axis of the movement vector R₁and selects the second axis as the principal axis of each of movementvectors R₂ to R₅. Hence, between the positions P₁ and P₂, the first axisserves as the principal axis; and between the positions P₂ to P₆, thesecond axis serves as the principal axis.

Meanwhile, the principal axis selecting unit 106 can also be configuredto perform correction in the form of increasing the inner product, fromamong the first inner product and the second inner product, which wascalculated using the principal axis of the previously-calculatedmovement vector. That is, the principal axis selecting unit 106 can beconfigured to set hysteresis for selecting the principal axis and toperform correction in such a way that, for example, if the first axiswas selected as the principal axis on the previous occasion, the samefirst axis is selected as the current principal axis without difficulty.

For example, the principal axis selecting unit 106 calculates aprincipal axis inner product, which points to the inner product of themovement vector and the axis selected as the principal axis on theprevious occasion, and calculates a non-principal axis inner product,which points to the inner product of the movement vector and the axisnot selected as the principal axis on the previous occasion. Then, theprincipal axis selecting unit 106 applies a correction function toincrement the principal axis inner product and compares thepost-correction principal axis inner product with the pre-correctionprincipal axis inner product. Accordingly, the principal axis selectingunit 106 selects the axis that is used in calculating the inner producthaving the greater modulus, from among the post-correction principalaxis inner product and the principal axis inner product, as theprincipal axis. Herein, for example, the correction function can be setto γ (γ>1.0). Moreover, consider the case when the first axis and thesecond axis intersect at a right angle. In that case, for example, ifthe slope of a movement vector with respect to the axis selected as theprincipal axis on the previous occasion is within the range of ±50°,then the principal axis selecting unit 106 can select that axis as theprincipal axis. On the other hand, if the slope of a movement vectorwith respect to the axis not selected as the principal axis on theprevious occasion is within the range of ±40°, then the principal axisselecting unit 106 can select that axis as the principal axis.

The turning point setting unit 108 sets positions at which there is achange in the principal axis as principal axis turning points anddirection turning points, and sets positions at which there is a changein the direction of the movement vectors in the principal axis componentas the direction turning points. For example, as illustrated in FIG. 4,since the principal axis changes from the first axis to the second axisat the position P₂, the turning point setting unit 108 sets the positionP₂ as a principal axis turning point and a direction turning point.Moreover, at the position P₅, in the component of the second axisserving as the principal axis, the direction of the movement vectorchanges from the right-hand direction to the left-hand direction. Thus,the turning point setting unit 108 sets the position P₅ as a directionturning point.

Meanwhile, as far as a change in the direction of any of the movementvectors in the principal axis component is concerned, the turning pointsetting unit 108 can determine that change with a change in the sign ofthe inner product of the movement vector and the principal axis.Moreover, at the positions P₃ and P₄, the direction of the movementvector changes in the component of the first axis that is not theprincipal axis, but not in the component of the second axis that is theprincipal axis. Hence, the turning point setting unit 108 does not setthe positions P₃ and P₄ as direction turning points.

The section setting unit 110 sets a determination target section, inwhich a start point is either set to the latest principal axis turningpoint when principal axis turning points are present or set to theinitial point indicating the initially-obtained position when principalaxis turning points are absent, and in which an end point is set to thelatest point that indicates the most-recently obtained position.Moreover, the section setting unit 110 sets a previous section, in whichan end point is set to the start point of the correspondingdetermination target section and in which a start point is either set tothe direction turning point that is present immediately before the endpoint when the direction turning points are present before the end pointor set to the initial point when no direction turning point is presentbefore the end point. Herein, the gesture portion set in thedetermination target section becomes the determination target fordetermining whether that gesture is in the direction along the firstaxis or in the direction along the second axis. Moreover, the gestureportion set in the previous section is used in determining whether ornot the gesture portion set in the determination target section isnoise, which points to unnecessary information that has no relation tothe gesture (such as waving a hand) intended by the photographic subject(user).

In the first embodiment, it is assumed that determination targetsections and the respective previous sections are set online, that is,in real time by the section setting unit 110. However, alternatively, itis also possible to set those sections by performing batch processing,that is, after the gesture is performed. Herein, for the sake ofconvenience, firstly, the explanation is given for the case when thedetermination target sections and the respective previous sections areset by performing batch processing. After that, the explanation is givenfor the case when those sections are set in real time.

FIG. 5 is an explanatory diagram for explaining an example of a methodof setting determination target sections and the respective previoussections by performing batch processing. In the example illustrated inFIG. 5, Q_(s) represents the initial position that is obtained by theobtaining unit 102 and that serves as the start point; Q₁ to Q₄represent principal axis turning points and direction turning points; K₁and K₂ represent direction turning points; and Q_(e) represents the lastposition that is obtained by the obtaining unit 102 and that serves asthe end point.

In the case of setting the determination target sections and therespective previous sections by performing batch processing; once thepoints Q_(s) to Q_(e) are obtained, the section setting unit 110 setsdetermination target sections and the respective previous sections inorder and sequentially outputs the information about those sections tothe determining unit 112. The information about a determination targetsection and the corresponding previous section can contain the positionsat both ends separating that determination target section from thecorresponding previous section, or can contain the timings of obtainingthose positions, or can contain IDs of those positions.

More particularly, firstly, the section setting unit 110 sets a sectionQ_(s) to Q₁ as the determination target section in which Q_(s) serves asthe start point and Q₁ serves as the end point; and sends theinformation about that determination target section to the determiningunit 112. In this case, the section setting unit 110 does not set anyprevious section. Subsequently, the section setting unit 110 sets asection Q₁ to Q₂ as the determination target section in which Q₁ servesas the start point and Q₂ serves as the end point. Besides, the sectionsetting unit 110 sets the section Q_(s) to Q₁ as the correspondingprevious section in which Q_(s) serves as the start point and Q₁ servesas the end point. Then, the section setting unit 110 sends theinformation about that determination target section and thecorresponding previous section to the determining unit 112. In anidentical manner, the section setting unit 110 sets a section Q₂ to Q₃as the determination target section and sets a section K₂ to Q₂ as thecorresponding previous section, and outputs that information to thedetermining unit 112. Similarly, the section setting unit 110 sets asection Q₃ to Q₄ as the determination target section and sets a sectionQ₂ to Q₃ as the corresponding previous section, and outputs thatinformation to the determining unit 112. Moreover, the section settingunit 110 sets a section Q₄ to Q_(e) as the determination target sectionand sets a section Q₃ to Q₄ as the corresponding previous section, andoutputs that information to the determining unit 112.

Thus, while setting the determination target sections and the respectiveprevious sections by batch processing; the section setting unit 110 setsdetermination target sections in each of which the start point is eitherset to the initial point or set to a principal axis turning point in asequential manner, and in each of which the end point is either set to aprincipal axis turning point present immediately after the correspondingstart point, when the principal axis turning points are presentsubsequent to the corresponding start point, or set to the last pointwhen no principal axis turning point is present subsequent to thecorresponding start point. Moreover, the section setting unit 110 setsprevious sections in each of which the end point is set to the startpoint of the subsequent determination target section, and in each ofwhich the start point is either set to the direction turning pointpresent immediately before that end point when the direction turningpoints are present before the end point or set to the initial point whenno direction turning point is present before the end point.

Explained below is the case in which the determination target sectionsand the respective previous sections are set in real time. While settingthe determination target sections and the respective previous sectionsin real time, every time the obtaining unit 102 obtains a position, thesection setting unit 110 sets a determination target section and thecorresponding previous section, and outputs the information about thosesections to the determining unit 112.

The specific explanation is given with reference to FIG. 4. Firstly,even when the obtaining unit 102 obtains the latest point (the initialpoint) P₁ that indicates the most-recently obtained position, thesection setting unit 110 neither sets a determination target section norsets a previous section.

Then, once the obtaining unit 102 obtains the latest point P₂, thesection setting unit 110 sets the section P₁ to P₂ as the determinationtarget section in which the start point is set to the initial point P₁and the end point is set to the latest point P₂, and outputs theinformation of that determination target section to the determining unit112. In this case, the section setting unit 110 does not set anyprevious section.

Subsequently, when the obtaining unit 102 obtains the latest point P₃,the position P₂ becomes the principal axis turning point. Hence, thesection setting unit 110 sets the section P₂ to P₃ as the determinationtarget section in which the start point is set to the latest principalaxis turning point P₂ and the end point is set to the latest point P₃.Besides, the section setting unit 110 sets the section P₁ to P₂ as thecorresponding previous section in which the start point is set to theinitial point P₁ and the end point is set to the principal axis turningpoint P₂. Then, the section setting unit 110 sends the information aboutthose two sections to the determining unit 112.

Subsequently, when the obtaining unit 102 obtains the latest point P₄,the section setting unit 110 sets the section P₂ to P₄ as thedetermination target section in which the start point is set to thelatest principal axis turning point P₂ and the end point is set to thelatest point P₄. Besides, the section setting unit 110 sets the sectionP₁ to P₂ as the corresponding previous section in which the start pointis set to the initial point P₁ and the end point is set to the principalaxis turning point P₂. Then, the section setting unit 110 sends theinformation about those two sections to the determining unit 112.

Subsequently, when the obtaining unit 102 obtains the latest point P₅,the section setting unit 110 sets the section P₂ to P₅ as thedetermination target section in which the start point is set to thelatest principal axis turning point P₂ and the end point is set to thelatest point P₅. Besides, the section setting unit 110 sets the sectionP₁ to P₂ as the corresponding previous section in which the start pointis set to the initial point P₁ and the end point is set to the principalaxis turning point P₂. Then, the section setting unit 110 sends theinformation about those two sections to the determining unit 112.

Subsequently, when the obtaining unit 102 obtains the latest point P₆,the section setting unit 110 sets the section P₂ to P₆ as thedetermination target section in which the start point is set to thelatest principal axis turning point P₂ and the end point is set to thelatest point P₆. Besides, the section setting unit 110 sets the sectionP₁ to P₂ as the corresponding previous section in which the start pointis set to the initial point P₁ and the end point is set to the principalaxis turning point P₂. Then, the section setting unit 110 sends theinformation about those two sections to the determining unit 112.

Then, the determining unit 112 calculates an evaluation value of thedetermination target section and an evaluation value of thecorresponding previous section and, based on those evaluation values,determines which of the first axis and the second axis is advantageousfor the movement of the specific part in the determination targetsection. More particularly, when the evaluation value of thedetermination target section exceeds the evaluation value of theprevious section, the determining unit 112 determines that the principalaxis in the determination target section is advantageous for themovement of the specific part in the determination target section. Onthe other hand, when the evaluation value of the determination targetsection is equal to or smaller than the evaluation value of the previoussection, the determining unit 112 determines that the principal axis inthe previous section is advantageous for the movement of the specificpart in the determination target section.

Herein, an evaluation value points to at least one of the following: theamount of change in the position or in the size of the specific part(i.e., the movement distance); the rate of change in the position or inthe size of the specific part (i.e., the movement velocity); and therate of acceleration in the position or in the size of the specific part(i.e., the movement acceleration). For example, an evaluation value canbe expressed as a function having the movement distance and the movementvelocity of the specific part as illustrated in Equation (1) given belowor can be expressed as a function having the movement distance, themovement velocity, and the movement acceleration of the specific part asillustrated in Equation (2) given below. Still alternatively, anevaluation value can be expressed as the movement distance of thespecific part, or the movement velocity of the specific part, or themovement acceleration of the specific part.

E=V×√{square root over (L)}  (1)

E=V×L×α  (2)

where, E represents the evaluation value, V represents the movementvelocity, L represents the movement distance, and α represents themovement acceleration.

The movement distance can be considered to be the sum of Euclideandistances of the movement vectors in the determination target section orcan be considered to be the sum of the principal axis components (sum ofthe modulus of the inner product of each movement vector and theprincipal axis) of the movement vectors. For example, in the exampleillustrated in FIG. 4, in the section P₂ to P₅ that serves as thedetermination target section at the point of time of obtaining thelatest point P₅, the movement distance can be considered to be the totaldistance calculated by adding the Euclidean distances of the movementvectors R₂ to R₄.

The movement velocity can be considered to be the maximum velocity amongthe velocities of the movement vectors in the determination targetsection or can be considered to be the average velocity of thevelocities of the movement vectors that are equal to or greater than apredetermined threshold value. Herein, if end points P_(b) and P_(c) atboth ends of a movement vector are obtained at timings T_(b) and T_(c),respectively; then the velocity of that movement vector can becalculated as |(distance between P_(b) and P_(c))/(T_(b)−T_(c))|. Forexample, in the example illustrated in FIG. 4, the movement velocity inthe section P₂ to P₅ can be considered to be the maximum velocity amongthe velocities of the movement vectors R₂ to R₄. If the end points P₂and P₃ at both ends of the movement vector R₂ are obtained at timings T₂and T₃, respectively; then the velocity of the movement vector R₂ can becalculated as |(Euclidean distance of the movement vector R₂)/(T₃−T₂)|.

Meanwhile, the method of calculating the movement velocity of thespecific part is not limited to the description given above. Forexample, the determining unit 112 can be configured to calculate thevelocities of pixels surrounding the positions of the specific part thatare obtained by the obtaining unit 102 (i.e., to calculate the opticalflow), and to set the average or the mode of the velocities of pixels asthe movement velocity of the specific part.

Alternatively, for example, by considering the position of the specificpart inside an image as an observation value and by considering theposition and the velocity of the specific part as state variables, thedetermining unit 112 can calculate the movement velocity of the specificpart with the use of a filter such as the Kalman filter that estimatesthe optimum state using a movement model of state variables such as anobservation value and a uniform motion. At that time, as the observationvalue, the determining unit 112 can further make use of the average ofthe velocities of pixels surrounding the positions of the specific part(i.e., the average of the optical flow).

The movement acceleration can be considered to be the maximum rate ofacceleration from among the rates of acceleration of the movementvectors in the determination target section or can be considered to bethe average of the rates of acceleration of the movement vectors thatare equal to or greater than a predetermined threshold value. Herein, ifthe end points P_(b) and P_(c) at both ends of a movement vector areobtained at the timings T_(b) and T_(c), respectively, and if the pointsP_(a) and P_(b) at both ends of the previous movement vector areobtained at timings T_(a) and T_(b), respectively; then the movementacceleration of that movement vector can be calculated as |(velocityfrom P_(b) to P_(c))−(velocity from P_(a) to P_(c))/(T_(b)−T_(c))|.

Explained below in detail with reference to FIG. 4 is the method ofdetermining the advantageous axis for the movement of the specific partin a determination target section. Firstly, when the section settingunit 110 sets the section P₁ to P₂ as the determination target section,no previous section exists. Hence, the determining unit 112 determinesthat the first axis serving as the principal axis in the section P₁ toP₂ is advantageous for the movement of the specific part.

Subsequently, the section setting unit 110 sets the section P₂ to P₃ asthe determination target section and sets the section P₁ to P₂ as thecorresponding previous section. In that case, the determining unit 112calculates the evaluation values of the section P₂ to P₃ and the sectionP₁ to P₂. Herein, it is assumed that the evaluation value of the sectionP₂ to P₃ (the determination target section) is equal to or smaller thanthe evaluation value of the section P₁ to P₂ (the corresponding previoussection). Thus, although the section P₂ to P₃ has the second axis as theprincipal axis, the determining unit 112 determines that the first axis,which is the principal axis in the section P₁ to P₂, is advantageous forthe movement of the specific part. That is, at that particular point oftime, the determining unit 112 considers the movement trajectory of thesection P₂ to P₃ as noise.

Subsequently, the section setting unit 110 sets the section P₂ to P₄ asthe determination target section and sets the section P₁ to P₂ as thecorresponding previous section. In that case, the determining unit 112calculates the evaluation values of the section P₂ to P₄ and the sectionP₁ to P₂. However, regarding the section P₁ to P₂, it is also possibleto use the evaluation value calculated on the previous occasion. Herein,it is assumed that the evaluation value of the section P₂ to P₄ (thedetermination target section) exceeds the evaluation value of thesection P₁ to P₂ (the corresponding previous section). Thus, thedetermining unit 112 determines that the second axis, which is theprincipal axis in the section P₂ to P₄, is advantageous for the movementof the specific part. That is, at that particular point of time, thedetermining unit 112 considers the movement trajectory of the section P₂to P₄ as a gesture.

In an identical manner, when each of the section P₂ to P₅ and thesection P₂ to P₆ is set as the determination target section, theevaluation value of the determination target section is assumed toexceed the evaluation value of the corresponding previous section.Hence, regarding each of those two determination target sections, thedetermining unit 112 determines that the second axis serving as theprincipal axis is advantageous for the movement of the specific part.

FIG. 6 is a diagram illustrating another example of a gesture performedby the specific part according to the first embodiment. In the exampleillustrated in FIG. 6, the specific part follows a trajectory in theright-hand direction (trajectories 261 and 262) and then follows atrajectory in the upward direction (trajectory 263). Meanwhile, in theexample illustrated in FIG. 6, the movement distance is used as theevaluation value.

During the trajectories 261 and 262, the trajectory in the right-handdirection serves as the determination target section, and there is noprevious section. Consequently, the determining unit 112 determines thatthe axis in the horizontal direction, which is the principal axis in thedetermination target section, is advantageous for the movement of thespecific part.

During the trajectory 263, the trajectory in the upward direction servesas the determination target section, and the trajectory in theright-hand direction serves as the corresponding previous section. Sincethe trajectory in the upward direction has a shorter movement distancethan the movement distance of the trajectory in the right-handdirection, the determining unit 112 determines that the evaluation valueof the trajectory in the right-hand direction exceeds the evaluationvalue of the trajectory in the upward direction. Accordingly, in thatdetermination target section too, the determining unit 112 determinesthat the axis in the horizontal direction, which is the principal axisin the corresponding previous section, is advantageous for the movementof the specific part.

As a result, any gesture in the vertical direction is considered asnoise, and the horizontal direction is determined to be advantageous asfar as the entire gesture is concerned.

The presenting unit 114 performs presentation according to thedetermination result of the presenting unit 114. For example, thepresenting unit 114 performs transition of the screen, which isdisplayed on a display, in the direction of the axis, from among thefirst axis and the second axis, which is determined by the determiningunit 112 to be advantageous for the movement of the specific part.

FIG. 7 is a diagram illustrating an example of presentation performed bythe presenting unit 114 according to the first embodiment. In theexample illustrated in FIG. 7, the presenting unit 114 displays anobject 181 on the left side of a screen 180, displays an object 182 inthe center of the screen 180, and displays an object 183 on the rightside of the screen 180. Herein, assume that the user performs a gestureof moving a hand 186, which represents the specific part, in a direction187, that is, in the left-hand direction. Regarding that gesture, sincethe determining unit 112 determines that the axis in the horizontaldirection is advantageous for the movement of the hand 186; thepresenting unit 114 performs transition of the objects in the horizontaldirection (to the left side) on the screen 180. As a result, on thescreen 180, the presenting unit 114 displays the object 182 on the leftside, the object 183 in the center, and an object 184 on the right side.

Meanwhile, if the recognition apparatus 100 includes a command issuingunit for issuing a command according to the determination resultobtained by the determining unit 112, then the presenting unit 114performs presentation according to the issued command.

For example, to the command issuing unit, the determining unit 112outputs the ID of the axis that is determined to be advantageous for themovement of the specific part. Herein, to the command issuing unit, thedetermining unit 112 can be configured to output the ID of the axis thatis determined to be advantageous for the movement of the specific partas well as output the information about the determination target sectionsuch as the IDs of both end points of the determination target section.Alternatively, only when there is a change in the axis that isdetermined to be advantageous for the movement of the specific part, thedetermining unit 112 can be configured to output the ID of the new axisto the command issuing unit.

Then, if the movement velocity of the specific part in that axiscomponent is equal to or greater than a threshold value, the commandissuing unit issues a command for the direction of that axis. Forexample, in the example illustrated in FIG. 4, when the section P₂ to P₃is the determination target section, the first axis becomes theadvantageous axis for the movement of the specific part in thedetermination target section. Therefore, if the velocity of the movementvector R₂ in the first axis component (i.e., the modulus of the innerproduct of the movement vector R₂ and the first axis) is equal to orgreater than a threshold value, that is, if the velocity component ofthe movement vector R₂ in the upward direction is equal to or greaterthan a threshold value; then the command issuing unit issues a commandfor the upward direction. In contrast, when the section P₂ to P₄ is thedetermination target section, the second axis becomes the advantageousaxis for the movement of the specific part in the determination targetsection. Therefore, if the velocity component in the right-handdirection of the greater velocity, of the velocities in the first axiscomponents of the movement vectors R₂ and R₃, is equal to or greaterthan a threshold value; then the command issuing unit issues a commandfor the right-hand direction.

According to the command that has been issued, the presenting unitperforms transition of the screen being displayed on the display.

FIG. 8 is a flowchart for explaining an exemplary sequence of operationsperformed by the recognition apparatus 100 according to the firstembodiment.

Firstly, the obtaining unit 102 obtains positions of a specific part ofa photographic subject that are represented in a coordinate systemhaving a first axis and a second axis (Step S101).

Then, with respect to the positions obtained by the obtaining unit 102,the calculating unit 104 calculates relative vectors from the respectivepreviously-obtained positions as movement vectors of the specific part(Step S102).

Subsequently, for each movement vector that is calculated by thecalculating unit 104, the principal axis selecting unit 106 calculates afirst inner product that is the inner product of the movement vector andthe first axis and calculates a second inner product that is the innerproduct of the movement vector and the second axis. Then, of the firstaxis and the second axis, the principal axis selecting unit 106 selectsthe axis that is used in calculating the inner product, from among thefirst inner product and the second inner product, having the greatermodulus as the principal axis (Step S103).

Subsequently, if the principal axis changes at the previously-obtainedposition obtained by the obtaining unit 102, the turning point settingunit 108 sets that position as a principal axis turning point and adirection turning point; and if the direction of the movement vectorchanges in the principal axis component at the previously-obtainedposition obtained by the obtaining unit 102, the turning point settingunit 108 sets that position as a direction turning point (Step S104).

Then, the section setting unit 110 sets a determination target section,in which a start point is either set to the latest principal axisturning point when principal axis turning points are present or set tothe initially-obtained position when principal axis turning points areabsent, and in which an end point is set to the latest point thatindicates the most-recently obtained position. Moreover, the sectionsetting unit 110 sets a previous section, in which an end point is setto the start point of the determination target section and in which astart point is either set to the direction turning point presentimmediately before the end point when the direction turning points arepresent before the end point or set to the initial point when nodirection turning point is present before the end point (Step S105).

Subsequently, the determining unit 112 calculates an evaluation value ofthe determination target section and an evaluation value of thecorresponding previous section and, based on those evaluation values,determines which of the first axis and the second axis is advantageousfor the movement of the specific part in the determination targetsection (Step S106).

Every time the obtaining unit 102 obtains the latest position of thespecific part of the photographic subject (Step S101), the operationsfrom Step S102 to Step S106 are repeated, and the presenting unit 114performs presentation according to the determination result obtained bythe determining unit 112.

FIG. 9 is a flowchart for explaining an exemplary sequence of operationsperformed by the turning point setting unit 108 in the recognitionapparatus 100 according to the first embodiment.

Firstly, the turning point setting unit 108 obtains, from the obtainingunit 102, a position P_(j) of the specific part in the latest frame aswell as obtains, from the principal axis selecting unit 106, a principalaxis ID of the latest frame (Step S120). The principal axis ID of thelatest frame points to the principal axis ID of the principal axis of amovement vector R_(i), which is the relative vector with respect to theposition P_(j) from a position P_(i) of the specific part in theprevious frame.

Then, the turning point setting unit 108 determines whether the positionP_(j) was obtained within two frames from the point of time when theobtaining unit 102 started obtaining the positions of the specific part(Step S121). If it is determined that the position P_(j) was obtainedwithin two frames from the point of time of starting to obtain thepositions of the specific part (Yes at Step S121), then the operationsare ended. That is because not enough positions are obtained to enablesetting of turning points.

On the other hand, if it is determined that the position P_(j) was notobtained within two frames from the point of time of starting to obtainthe positions of the specific part (No at Step S121), then the turningpoint setting unit 108 obtains, from the obtaining unit 102, theposition P_(i) of the specific part in the previous frame and a positionP_(h) of the specific part in the last but one frame; as well asobtains, from the principal axis selecting unit 106, the principal axisID of the previous frame (Step S122). The principal axis ID of theprevious frame points to the principal axis ID of the principal axis ofa movement vector R_(h), which is the relative vector with respect tothe position P_(i) from the position P_(h) of the specific part in thelast but one frame. Meanwhile, alternatively, the principal axis ID ofthe latest frame that is obtained in the previous frame can be storedand used by the turning point setting unit 108 as the principal axis IDof the previous frame.

Subsequently, the turning point setting unit 108 determines whether ornot the principal axis ID of the latest frame is different than theprincipal axis ID of the previous frame (Step S123).

When the principal axis ID of the latest frame is different than theprincipal axis ID of the previous frame (Yes at Step S123), the turningpoint setting unit 108 sets the position P_(i) as a principal axisturning point and a direction turning point (Step S124). That marks theend of the operations.

On the other hand, when the principal axis ID of the latest frame issame as the principal axis ID of the previous frame (No at Step S123),the turning point setting unit 108 determines whether or not themovement trajectory (movement vector) of the specific part at theposition P_(i) is changing direction in the principal axis component(Step S125). More particularly, the turning point setting unit 108determines whether or not the inner product of the movement vector R_(i)and the principal axis of the latest frame has a different sign than thesign of the inner product of the movement vector R_(j) and the principalaxis of the previous frame. However, if the inner product of themovement vector and the principal axis of the previous frame is equal tozero, it is understood that there was no change in the principal axiscomponent at the movement vector R_(j). In that case, the turning pointsetting unit 108 obtains a position P_(g) of the specific part that isobtained three times earlier; and then makes use of the inner productbetween a movement vector R_(g), which is the relative vector withrespect to the position P_(h) from the position P_(g) of the specificpart in the last frame but two, and the principal axis of the last framebut two.

When the movement trajectory (movement vector) of the specific part atthe position P_(i) is changing direction in the principal axis component(Yes at Step S125), that is, when both inner products have differentsigns; the turning point setting unit 108 sets the position P_(i) as adirection turning point (Step S126). That marks the end of theoperations.

On the other hand, when the movement trajectory (movement vector) of thespecific part at the position P_(i) is not changing direction in theprincipal axis component (No at Step S125), that is, when both innerproducts have the same sign; the operations are ended.

FIG. 10 is a flowchart for explaining an exemplary sequence ofoperations performed by the determining unit 112 in the recognitionapparatus 100 according to the first embodiment.

Firstly, the determining unit 112 checks whether or not a previoussection is present (Step S130).

When a previous section is present (Yes at Step S130), the determiningunit 112 calculates an evaluation value E_(c) of the determinationtarget section and an evaluation value E_(b) of the previous section,and determines whether or not the evaluation value E_(c) is equal to orsmaller than the evaluation value E_(b) (Step S131).

If the evaluation value E_(c) is equal to or smaller than the evaluationvalue E_(b) (Yes at Step S131), the determining unit 112 determines thatthe principal axis in the previous section is advantageous for themovement of the specific part in the determination target section (StepS132). That marks the end of the operations.

Meanwhile, if a previous section is absent (No at Step S130) or if theevaluation value E_(c) is not equal to or smaller than the evaluationvalue E_(b) (No at Step S131), then the determining unit 112 determinesthat the principal axis in the determination target section isadvantageous for the movement of the specific part in the determinationtarget section (Step S133). That marks the end of the operations.

As described above, in the first embodiment, the gesture in adetermination target section is recognized using the correspondingprevious section. Thus, according to the first embodiment, it becomespossible to determine whether or not the gesture in a determinationtarget section is noise. Because of that, even if a movementrepresenting noise is included in a gesture, such movement can berecognized correctly thereby enabling prevention of a decline in therecognition accuracy.

For example, if a movement representing noise is included at the end ofa gesture, then the portion of the normal gesture represents theprevious section and the noise portion represents the determinationtarget section. There, by referring to the previous section, it becomespossible to determine that the gesture in the determination targetsection is noise and to make substitution with the movement direction ofthe normal gesture in the previous section. In this way, even when amovement representing noise is included in a gesture, the noise can berecognized in a correct manner.

Second Embodiment

In a second embodiment, the explanation is given for an example ofrecognizing the gesture in a determination target section by making useof the corresponding subsequent section. In the following explanation,the focus is on elaborating the differences with the first embodiment.Moreover, the constituent elements having identical functions to thefunctions described in the first embodiment are referred to by the samenames and reference numerals, and the explanation thereof is notrepeated.

FIG. 11 is a configuration diagram illustrating an example of arecognition apparatus 200 according to the second embodiment. In therecognition apparatus 200 according to the second embodiment; a sectionsetting unit 210, a determining unit 212, and a presenting unit 214 aredifferent as compared to the first embodiment.

The section setting unit 210 sets a determination target section, inwhich a start point is either set to the latest principal axis turningpoint when principal axis turning points are present or set to theinitially-obtained position when principal axis turning points areabsent, and in which an end point is either set to the principal axisturning point present immediately after the start point, when theprincipal axis turning points are present subsequent to the start pointbut before the elapse in a predetermined time period since the settingof the start point, or set to the latest point that indicates themost-recently obtained position before the elapse in a predeterminedtime period, when no principal axis turning point is present subsequentto the start point but before the elapse in the predetermined time.Moreover, the section setting unit 210 sets a subsequent section, inwhich a start point is set to the end point of the determination targetsection and in which an end point is either set to the direction turningpoint present immediately after to the start point when the directionturning points are present subsequent to the start point or set to thelatest point when the latest point is present subsequent to the startpoint. Moreover, the gesture portion set in the subsequent section isused in determining whether or not the gesture portion set in thedetermination target section is noise.

In the second embodiment too, it is assumed that determination targetsections and the respective subsequent sections are set online, that is,in real time by the section setting unit 210. However, alternatively, itis also possible to set those sections by performing batch processing,that is, after the gesture is performed. Herein, for the sake ofconvenience, firstly, the explanation is given for the case whendetermination target sections and the respective subsequent sections areset by performing batch processing. Then, the explanation is given forthe case when those sections are set in real time.

Explained below with reference to FIG. 5 is the setting of determinationtarget sections and the respective subsequent sections by performingbatch processing. While setting the determination target sections andthe respective subsequent sections by performing batch processing, afterthe initial position Q_(s) and the last position Q_(e) are obtained, thesection setting unit 210 sets the determination target sections and therespective subsequent sections in order, and sequentially outputs to thedetermining unit 212 the information regarding the determination targetsections and the respective subsequent sections that have been set. Theinformation regarding a determination target section and thecorresponding subsequent section that have been set can contain thepositions at both ends separating that determination target section fromthe corresponding subsequent section, or can contain the timings ofobtaining those positions, or can contain IDs of those positions.

More particularly, firstly, the section setting unit 210 sets thesection Q_(s) to Q₁ as a determination target section in which Q_(s)serves as the start point and Q₁ serves as the end point; and sets asection Q₁ to K₁ as the subsequent section in which Q₁ serves as thestart point and K₁ serves as the end point. Then, the section settingunit 210 sends the information about those sections to the determiningunit 212. Subsequently, the section setting unit 210 sets the section Q₁to Q₂ as a determination target section in which Q₁ serves as the startpoint and Q₂ serves as the end point; and sets a section Q₂ to Q₃ as thesubsequent section in which Q₂ serves as the start point and Q₃ servesas the end point. Then, the section setting unit 210 sends theinformation about those sections to the determining unit 212. In anidentical manner, the section setting unit 210 sets the section Q₂ to Q₃as the determination target section, sets a section Q₃ to Q₄ as thesubsequent section, and sends that information to the determining unit212. Moreover, the section setting unit 210 sets the section Q₃ to Q₄ asthe determination target section, sets a section Q₄ to Q₅ as asubsequent section, and sends that information to the determining unit212. Lastly, the section setting unit 210 sets the section Q₄ to Q₅ asthe determination target section and, without setting a subsequentsection, sends that information to the determining unit 212.

Thus, while setting the determination target sections and the respectivesubsequent sections by performing batch processing; the section settingunit 210 sets determination target sections in each of which the startpoint is either set to the initial point or set to a principal axisturning point in a sequential manner, and in each of which the end pointis either set to a principal axis turning point present immediatelyafter the start point, when the principal axis turning points arepresent subsequent to the corresponding start point, or set to the lastpoint when no principal axis turning point is present subsequent to thecorresponding start point. Moreover, the section setting unit 210 setssubsequent sections in each of which the start point is set to the endpoint of the corresponding determination target section, and in each ofwhich the end point is either set to the direction turning point presentimmediately after the corresponding start point when the directionturning points are present subsequent to the corresponding start pointor set to the last point when no direction turning point is presentsubsequent to the corresponding start point.

Given below is the explanation regarding setting a determination targetsection and the corresponding subsequent section in real time. Whilesetting the determination target sections and the respective subsequentsections in real time, the section setting unit 210 requires a maximumpredetermined time period from the start of setting a determinationtarget section up to the finalization of that determination targetsection and the corresponding subsequent section. That is, since thestart of setting a determination target section at the timing T_(i), thesection setting unit 210 finalizes the determination target section andthe corresponding subsequent section before the elapse in apredetermined time period T_(0a), that is, until the point of time ofT_(i)+T_(0a). Thus, in the second embodiment, although it is assumedthat a determination target section and the corresponding subsequentsection are set in real time; in a precise sense, a delay within apredetermined time period occurs before a determination target sectionand the corresponding subsequent section is set. With that, thedetermination performed by the determining unit 212 (described later)also gets delayed within a predetermined time period.

Moreover, in the second embodiment, the predetermined time period T_(0a)is set to 0.4 seconds. However, that is not the only possible case.Alternatively, depending on the movement velocity of the specific part,or the specific bodily part, in a determination target section, thesection setting unit 210 can change the predetermined time periodT_(0a). For example, the predetermined time period T_(0a) can be set todecrease with an increase in the movement velocity.

The specific explanation is given with reference to FIG. 4. Firstly,even after the obtaining unit 102 obtains the latest point (the initialpoint) P₁ that indicates the latest position, the section setting unit210 does neither sets a determination target section nor sets asubsequent section.

When the obtaining unit 102 obtains the latest point P₂ (assumed to beat a timing T₂), the section setting unit 210 sets the section P₁ to P₂,in which the initial point P₁ is the start point and in which the latestpoint P₂ is the end point at the timing T₂, as the determination targetsection. At that point of time, the scope of the determination targetsection is yet to be finalized. If another latest point is obtainedafter the timing T₂ but before the elapse in the time period T_(0a),then the end point of the determination target section may be updated.However, in the example illustrated in FIG. 4, since the position P₂serves as a principal axis turning point present immediately after thestart point, it remains to be the end point of the determination targetsection.

For example, if the latest point P₃ is obtained after the timing T₂ butbefore the elapse in the time period T_(0a), then the position P₂becomes the principal axis turning point present immediately after thestart point. Hence, the section setting unit 210 finalizes the sectionP₁ to P₂, which has the principal axis turning point P₂ as the endpoint, as the determination target section. Besides, in this case, thesection setting unit 210 sets the section P₂ to P₃, which has theprincipal axis turning point P₂ as the start point and the latest pointP₃ as the end point, as the subsequent section. Then, as the informationabout the determination target section when the timing T₂ is the settingstart timing, the section setting unit 210 sends to the determining unit212 the information about the determination target section and thecorresponding subsequent section that have been set (finalized).

Besides, for example, even in the case when the latest points P₃ to P₆are obtained after the timing T₂ but before the elapse in the timeperiod T_(0a), the position P₂ becomes the principal axis turning pointpresent immediately after the start point. Hence, the section settingunit 210 finalizes the section P₁ to P₂, which has the principal axisturning point P₂ as the end point, as the determination target section.Moreover, in this case, the section setting unit 210 sets the section P₂to P₅, which has the direction turning point P₅ that is the directionturning point present immediately after the start point as the endpoint, as the corresponding subsequent section. Then, as the informationabout the determination target section when the timing T₂ is the settingstart timing, the section setting unit 210 to the determining unit 212the information about the determination target section and thesubsequent section that have been set (finalized).

Meanwhile, for example, if the latest point P₃ is not obtained after thetiming T₂ and before the elapse in the time period T_(0a), then theposition P₂ becomes the latest point. Hence, the section setting unit210 finalizes the section P₁ to P₂, which has the latest point P₂ as theend point, as the determination target section. In this case, thesection setting unit 210 does not set a subsequent section. Then, as theinformation about the determination target section when the timing T₂ isthe setting start timing, the section setting unit 210 sends to thedetermining unit 212 the information about the determination targetsection that has been finalized.

Subsequently, when the obtaining unit 102 obtains the latest point P₂(assumed to be at a timing T₃), the section setting unit 210 sets thesection P₂ to P₃, in which the latest principal axis turning point P₂ isthe start point and in which the latest point P₃ is the end point at thetiming T₃, as the determination target section. Herein, if the latestpoint P₄ is obtained after the timing T₃ but before the elapse in thetime period T_(0a), the section setting unit 210 updates the end pointto the latest point P₄ because no principal axis turning point ispresent subsequent to the principal axis turning point P₂. Thus, thesection setting unit 210 updates the determination target section to thesection P₂ to P₄. In this case, the section setting unit 210 does notset a subsequent section. Then, as the information about thedetermination target section when the timing T₃ is the setting starttiming, the section setting unit 210 sends to the determining unit 212the information about the updated determination target section.

Until the obtaining unit 102 obtains the latest point P₆, the identicaloperation is performed.

Meanwhile, instead of outputting to the determining unit 212 theinformation about a determination target section and the correspondingsubsequent section only after those sections are finalized, the sectionsetting unit 210 can keep on outputting to the determining unit 212 theinformation about a determination target section and the correspondingsubsequent section during the period from the start of setting thedetermination target section to the finalization of those sections.

Then, the determining unit 212 calculates an evaluation value of thedetermination target section and an evaluation value of thecorresponding subsequent section and, based on those evaluation values,determines which of the first axis and the second axis is advantageousfor the movement of the specific part in the determination targetsection. More particularly, when the evaluation value of thedetermination target section exceeds the evaluation value of thecorresponding subsequent section, the determining unit 212 determinesthat the principal axis in the determination target section isadvantageous for the movement of the specific part in the determinationtarget section. On the other hand, when the evaluation value of thedetermination target section is equal to or smaller than the evaluationvalue of the corresponding subsequent section, the determining unit 212determines that the principal axis in the subsequent section isadvantageous for the movement of the specific part in the determinationtarget section.

For example, while determining the advantageous axis for the movement ofthe specific part at a desired determination timing T, the sectionsetting unit 210 starts the setting of a determination target section atthe desired determination timing T and finalizes the determinationtarget section before the elapse in the predetermined time period T_(0a)from the desired determination timing T. Then, at that point of time(i.e., at the point of time of elapse in the predetermined time periodT_(0a)), the determining unit 212 finalizes the axis that isadvantageous for the movement of the specific part in the determinationtarget section at the desired determination timing T. Herein, thedesired determination timing points to the setting start timing forsetting a determination target section by the section setting unit 210.

FIG. 12 is a diagram illustrating another example of a gesture performedby the specific part according to the second embodiment. In the exampleillustrated in FIG. 12, the specific part follows a trajectory in theupward direction (trajectory 261) and then follows a trajectory in theright-hand direction (trajectories 262 and 263). Meanwhile, in theexample illustrated in FIG. 12, the movement distance is used as theevaluation value.

During the trajectory 261 (hereinafter, referred to as “timing T₂₆₁”),the trajectory in the upward direction serves as the determinationtarget section. Herein, assume that, before the elapse in thepredetermined time period T_(0a), the trajectory changes from thetrajectory 262 to the trajectory 263. In that case, as far as theinformation about a determination target section and the correspondingsubsequent section is concerned at the time when the desireddetermination timing (i.e., the setting start timing for setting adetermination target section) is the timing T₂₆₁; during the trajectory263, that is, at the timing obtained as (timing T₂₆₁)+(timing T_(0a)),the concerned information indicates that the trajectory in the upwarddirection still serves as the determination target section and thetrajectory in the right-hand direction serves as the correspondingsubsequent section. Since the movement distance of the trajectory in theright-hand direction is exceeding the movement distance of thetrajectory in the upward direction, the determining unit 212 determinesthat the evaluation value of the trajectory in the right-hand directionis higher than the evaluation value of the trajectory in the upwarddirection. Accordingly, the determining unit 212 determines that theaxis in the horizontal direction, which is the principal axis in thesubsequent section, is advantageous for the movement of the specificpart in the determination target section. As a result, any gesture inthe vertical direction is considered as noise, and the horizontaldirection is determined to be advantageous as far as the entire gestureis concerned.

Meanwhile, in the case when the section setting unit 210 keeps onoutputting the information about a determination target section and thecorresponding subsequent section; the determining unit 212 can keep ondetermining the axis, which is advantageous for the movement of thespecific part in the determination target section, for a predeterminedtime period until the determination target section and the subsequentsection are finalized. In this case, the determining unit 212 can alsocalculate the progress rate of the predetermined time. Herein, theprogress rate can be a value obtained by interpolation to 0 of thetiming T_(i) that is the point of time of starting the setting of adetermination target section, interpolation to 1 of the timingT_(i)+T_(0a) that is the point of time of elapse in the predeterminedtime period T_(0a), and linear interpolation of the period from thetiming T_(i) to the timing T_(i)+T_(0a). Thus, for a timing t whereT_(i)≦t≦T_(i)+T_(0a); the progress rate is (t−T_(i))/T_(0a).

FIG. 13 is a diagram for illustrating an example of axes each beingadvantageous at a particular progress rate determined by the determiningunit 212 according to the second embodiment. In the example illustratedin FIG. 13, when the desired determination timing is the timing T_(i),the first axis is advantageous for the movement of the specific part inthe determination target section during the timing T_(i) to the timingT_(i)+20 (i.e., during the progress rate of 0 to 0.30) and the secondaxis is advantageous for the movement of the specific part in thedetermination target section during the timing T_(i)+30 to the timingT_(i)+70 (i.e., during the progress rate of 0.45 to 1.00). As a result,when the desired determination timing is the timing T_(i), the secondaxis is finalized as the axis that is advantageous for the movement ofthe specific part in the determination target section.

Meanwhile, the progress rate can be based on the timing of obtainingeach position that is obtained by the obtaining unit 102. Moreover, onlyin the case when the advantageous axis for the movement of the specificpart is determined for a particular determination target section, thedetermining unit 212 can be configured to keep on determining theadvantageous axis and calculating the progress rate of a predeterminedtime period. Herein, a particular determination target section eitherpoints to a determination target section to be subjected to thedetermination of the advantageous axis immediately after an advantageousaxis has been finalized, or points to a determination target section inwhich the movement vector has the movement velocity or the movementdistance equal to or greater than a threshold value.

FIG. 14 is a diagram for illustrating another example of axes each beingadvantageous at a particular progress rate determined by the determiningunit 212 according to the second embodiment. In the example illustratedin FIG. 14, sequential determination of advantageous axes andcalculation of progress rates in predetermined time periods is performedonly when the axis advantageous to the movement of the specific part isdetermined for a determination target section that is to be subjected tosuch determination immediately after an advantageous axis has beenfinalized.

During the timing T_(i) to the timing T_(i)+70 (i.e., during theprogress rate 0 to 1.00), the condition is identical to that illustratedin FIG. 13. Regarding the determination target section for which thedetermination of advantageous axis is started at the timing T_(i), thesecond axis is finalized as the advantageous axis at the timingT_(i)+70. For that reason, regarding the determination target sectionfor which the determination of advantageous axis is started at thetiming T_(i)+80, the determining unit 212 performs sequentialdetermination of the advantageous axis and calculates the progress rateof a predetermined time period.

In the determination target section during the timing T_(i)+80 to thetiming T_(i)+150 (i.e., during the progressive rate of 0 to 1.00), thesecond axis is the advantageous axis for the movement of the specificpart. As a result, the second axis is finalized as the axis that isadvantageous for the movement of the specific part in the determinationtarget section for which the determination of the advantageous axis isstarted at the timing T_(i)+80.

Regarding the determination target section for which the determinationof the advantageous axis is started at the timing T_(i)+80, the secondaxis is finalized to be the advantageous axis at the timing T_(i)+150.Hence, regarding the determination target section for which thedetermination of advantageous axis is started at the timing T_(i)+160,the determining unit 212 performs sequential determination of theadvantageous axis and calculates the progress rate of a predeterminedtime period.

When the determining unit 212 performs sequential determination, withina predetermined time period, of the advantageous axis for the movementof the specific part in a determination target section, the presentingunit 214 can be configured to further perform supplementary presentationwith a predetermined time period, that is, depending on thedetermination result before the determination target section and thecorresponding subsequent section are finalized. In the secondembodiment, when the setting of a determination target section arestarted at the timing T_(i), the advantageous axis for the movement ofthe specific part in that determination target section is finalized bythe timing T_(i)+T_(0a). Therefore, the presenting unit 214 performssupplementary presentation depending on the determination resultobtained by the determining unit 212 between the timings T_(i) andT_(i)+T_(0a). After the timing T_(i)+T_(0a), by which the advantageousaxis for the movement of the specific part is finalized, thepresentation performed is identical to that described in the firstembodiment.

FIG. 15 is an example of the supplementary presentation performed by thepresenting unit 214 according to the second embodiment. For example,with reference to FIG. 13, at the timing T_(i)+30, the advantageous axisfor the movement of the specific part in a determination target sectionfor which the settings are started at the timing T_(i) changes from thefirst axis to the second axis. At that point of time, with reference tothe screen 180 illustrated in FIG. 7 in which the object 181 isdisplayed on the left side, the object 182 is displayed in the center,and the object 183 is displayed on the right side; the presenting unit214 can display a screen 280 as illustrated in FIG. 15 in which theobjects 181 to 183 are tilted in the horizontal direction (in theleft-hand direction). Then, at the timing T_(i)+70, once the second axisis finalized as the advantageous axis for the movement of the specificpart in a determination target section for which such determination isstarted at the time timing T_(i); the presenting unit 214 changes thescreen 280 to the screen 180 illustrated in FIG. 7 in which the object181 is displayed on the left side, the object 182 is displayed in thecenter, and the object 183 is displayed on the right side. As a result,it becomes possible to notify the user about the axis that is beingdetermined as the advantageous axis.

FIGS. 16 and 17 are diagrams illustrating other examples of thesupplementary presentation performed by the presenting unit 214according to the second embodiment. With reference to the screen 180illustrated in FIG. 7 in which the object 181 is displayed on the leftside, the object 182 is displayed in the center, and the object 183 isdisplayed on the right side; the presenting unit 214 can be configuredto display, not the screen 280 illustrated in FIG. 15, but a screen 281in which the objects 181 to 183 are rotated in horizontal direction (theleft-hand direction) or a screen 282 in which an arrow 283 isadditionally displayed to indicate the horizontal direction (theleft-hand direction).

FIG. 18 is a flowchart for explaining an exemplary sequence ofoperations performed in the recognition apparatus 200 according to thesecond embodiment.

Herein, the operations from Steps S201 to S204 are identical to theoperations from Steps S101 to S104 in the flowchart illustrated in FIG.8.

Subsequently, the section setting unit 210 sets a determination targetsection, in which a start point is either set to the latest principalaxis turning point when principal axis turning points are present or setto the initially-obtained position when principal axis turning pointsare absent, and in which an end point is either set to the principalaxis turning point present immediately after the start point, when theprincipal axis turning points are present subsequent to the start pointbut before the elapse in a predetermined time period since the settingof the start point, or set to the latest point that indicates themost-recently obtained position before the elapse in a predeterminedtime period, when no principal axis turning point is present subsequentto the start point but before the elapse in the predetermined time.Moreover, the section setting unit 210 sets a subsequent section, inwhich a start point is set to the end point of the abovementioneddetermination target section and in which an end point is either set tothe direction turning point present immediately after the start pointwhen such the direction turning points are present subsequent to thestart point or set to the latest point when the latest point is presentsubsequent to the start point (Step S205).

Then, the determining unit 212 calculates an evaluation value of thedetermination target section and an evaluation value of thecorresponding subsequent section and, based on those evaluation values,determines which of the first axis and the second axis is advantageousfor the movement of the specific part in the determination targetsection (Step S206).

Every time the obtaining unit 102 obtains the latest position of thespecific part of the photographic subject (Step S201), the operationsfrom Step S202 to Step S206 are performed, and the presenting unit 214performs presentation according to the determination result obtained bythe determining unit 212.

FIG. 19 is a flowchart for explaining an exemplary sequence ofoperations performed by the determining unit 212 in the recognitionapparatus 200 according to the second embodiment.

Firstly, the determining unit 212 checks whether or not a subsequentsection is present (Step S230).

When a subsequent section is present (Yes at Step S230), the determiningunit 212 calculates an evaluation value E_(c) of the determinationtarget section and an evaluation value E_(a) of the subsequent section,and determines whether or not the evaluation value E_(c) is equal to orsmaller than the evaluation value E_(a) (Step S231).

If the evaluation value E_(c) is equal to or smaller than the evaluationvalue E_(a) (Yes at Step S231), the determining unit 212 determines thatthe principal axis in the subsequent section is advantageous for themovement of the specific part in the determination target section (StepS232). That marks the end of the operations.

Meanwhile, if a subsequent section is absent (No at Step S230) or if theevaluation value E_(c) is not equal to or smaller than the evaluationvalue E_(a) (No at Step S231), the determining unit 212 determines thatthe principal axis in the determination target section is advantageousfor the movement of the specific part in the determination targetsection (Step S233). That marks the end of the operations.

In the recognition apparatus 200, there are times when a determinationtarget section and the corresponding subsequent section get finalizedbefore the elapse in a predetermined time period since the start ofsetting of the determination target section, and there are times whenthe advantageous axis for the movement of the specific part in thedetermination target section gets finalized before the elapse in apredetermined time period. For example, before the elapse in apredetermined time period since the start of setting of a determinationtarget section, if a principal axis turning point is present subsequentto the start point of the determination target period and if a directionturning point is present subsequent to the principal axis turning point;then the determination target section and the corresponding subsequentsection get finalized before the elapse in a predetermined time period.Moreover, before the elapse in a predetermined time period since thestart of setting of a determination target section, if the evaluationvalue of the subsequent section exceeds the evaluation value of thedetermination target section, the advantageous axis for the movement ofthe specific part in the determination target section gets finalized tothe principal axis of the subsequent section. That eliminates the needof waiting for the elapse in the predetermined time period. Thus, thedetermining unit 212 can be configured to take these points into accountwhile performing the operations.

FIG. 20 is a flowchart for explaining another exemplary sequence ofoperations performed by the determining unit 212 in the recognitionapparatus 200 according to the second embodiment.

Firstly, the determining unit 212 calculates a difference between acurrent timing T and a desired determination timing T_(c) that is thetiming of starting the setting of a determination target section (StepS241), and determines whether or not a predetermined time period T_(0a)has elapsed since the desired determination timing T_(e) (Step S242).

If the predetermined time period T_(0a) has not elapsed since thedesired determination timing T_(c) (No at Step S242), the determiningunit 212 determines whether or not a subsequent section is present (StepS243). If a subsequent section is absent (No at Step S243), the systemcontrol returns to Step S241.

On the other hand, when a subsequent section is present (Yes at StepS243), the determining unit 212 calculates an evaluation value E_(c) ofthe determination target section and an evaluation value E_(a) of thesubsequent section, and determines whether or not the evaluation valueE_(a) is exceeding the evaluation value E_(c) (Step S244).

If the evaluation value E_(a) is exceeding the evaluation value E_(c)(Yes at Step S244), the determining unit 212 determines that theprincipal axis in the subsequent section is advantageous for themovement of the specific part in the determination target section (StepS247). That marks the end of the operations.

In contrast, if the evaluation value E_(a) is not greater than theevaluation value E_(c) (No at Step S244), the determining unit 212checks whether or not the subsequent section has been finalized (StepS245). More particularly, the determining unit 212 checks whether or notthe end point of the subsequent section is a direction turning point.

If the subsequent section has been finalized (Yes at Step S245), thatis, if the end point of the subsequent section is a direction turningpoint; because of the fact that the evaluation value E_(a) does notexceed the evaluation value E_(c), the determining unit 212 determinesthat the principal axis in the determination target section isadvantageous for the movement of the specific part in the determinationtarget section (Step S246). That marks the end of the operations.

On the other hand, if the subsequent section has not been finalized (Noat Step S245), that is, if the end point of the subsequent section isnot a direction turning point; the system control returns to Step S241.

Hence, when the predetermined time period T_(0a) has elapsed since thedesired determination timing T_(c) (Yes at Step S242), either asubsequent section is absent or the evaluation value E_(a) does notexceed the evaluation value E_(c). In that case, the determining unit212 determines that the principal axis in the determination targetsection is advantageous for the movement of the specific part in thedetermination target section (Step S246). That marks the end of theoperations.

Meanwhile, when a subsequent section is present (Yes at Step S243), itmeans that the determination target section has been finalized. Thatenables the usage of the evaluation value E_(c) that has been alreadycalculated. Thus, the evaluation value E_(c) need not be calculatedagain.

As described above, in the second embodiment, the gesture in adetermination target section is recognized using the correspondingsubsequent section. Thus, according to the second embodiment, it becomespossible to determine whether or not the gesture in a determinationtarget section is noise. Because of that, even if a movementrepresenting noise is included in a gesture, such movement can berecognized correctly thereby enabling prevention of a decline in therecognition accuracy.

For example, if a movement representing noise is included at the startof a gesture, then the noise portion represents the determination targetsection and the portion of the normal gesture represents the subsequentsection. There, by referring to the subsequent section, it becomespossible to determine that the gesture in the determination targetsection is noise and to make substitution with the movement direction ofthe normal gesture in the subsequent section. In this way, even when amovement representing noise is included in a gesture, the noise can berecognized in a correct manner.

Third Embodiment

In a third embodiment, the explanation is given for an example in whichthe gesture in a determination target section is recognized by makinguse of the corresponding previous section as well as the correspondingsubsequent section; and in which the direction indicated along theadvantageous axis, which is advantageous for the movement of thespecific part, or the specific bodily part, in the determination targetsection, is also recognized. In the following explanation, the focus ison elaborating the differences with the first embodiment. Moreover, theconstituent elements having identical functions to the functionsdescribed in the first embodiment are referred to by the same names andreference numerals, and the explanation thereof is not repeated.

FIG. 21 is a configuration diagram illustrating an example of arecognition apparatus 300 according to the third embodiment. In therecognition apparatus 300 according to the third embodiment; a sectionsetting unit 310, a determining unit 312, and a presenting unit 314 aredifferent as compared to the first embodiment. Besides, the recognitionapparatus 300 further includes a first axis determining unit 316, asecond axis determining unit 318, and a determination result selectingunit 320.

The section setting unit 310 has the combined functions of the sectionsetting unit 110 according to the first embodiment and the sectionsetting unit 210 according to the second embodiment.

That is, the section setting unit 310 sets a determination targetsection, in which a start point is either set to the latest principalaxis turning point when principal axis turning points are present or setto the initially-obtained position when principal axis turning pointsare absent, and in which an end point is set to the latest point thatindicates the most-recently obtained position. Moreover, the sectionsetting unit 310 sets a previous section, in which an end point is setto the start point of the determination target section and in which astart point is either set to the direction turning point presentimmediately before the end point when the direction turning points arepresent before the end point or set to the initial point when nodirection turning point is present before the end point. At this pointof time, the previous section does get finalized but the determinationtarget section is not yet finalized.

Then, the section setting unit 310 updates the end point either to theprincipal axis turning point present immediately after the start point,when the principal axis turning points are present subsequent to thestart point but before the elapse in a predetermined time period sincethe setting of the start point, or to the latest point that indicatesthe most-recently obtained position after the elapse in a predeterminedtime period, when no principal axis turning point is present subsequentto the start point but before the elapse in the predetermined time.Moreover, the section setting unit 310 sets a subsequent section, inwhich a start point is set to the end point of the updated determinationtarget section and in which an end point is either set to the directionturning point present immediately after the start point when thedirection turning points are present subsequent to the start point orset to the latest point present after the elapse in a predetermined timeperiod when latest points are present subsequent to the start point.

As described above, the section setting unit 310 has the combinedfunctions of the section setting unit 110 according to the firstembodiment and the section setting unit 210 according to the secondembodiment. Thus, the determining unit 312 calculates an evaluationvalue of the determination target section, an evaluation value of thecorresponding previous section, and an evaluation value of thecorresponding subsequent section; and, based on those evaluation values,determines which of the first axis and the second axis is advantageousfor the movement of the specific part in the determination targetsection.

More particularly, if the evaluation value of the determination targetsection is exceeding the evaluation value of the previous section or ifthe previous section is absent, as well as if the evaluation value ofthe determination target section is exceeding the evaluation value ofthe subsequent section or if the subsequent section is absent; thedetermining unit 312 determines that the principal axis in thedetermination target section is advantageous for the movement of thespecific part in the determination target section.

When the evaluation value of the determination target section is equalto or smaller than the evaluation value of the previous section, thedetermining unit 312 determines that the principal axis in the previoussection is advantageous for the movement of the specific part in thedetermination target section. As a result, without having to wait forthe elapse in a predetermined time period, the principal axis in theprevious section can be determined to be advantageous for the movementof the specific part in the determination target section.

When the evaluation value of the determination target section is equalto or smaller than the evaluation value of the subsequent section, thedetermining unit 312 determines that the principal axis in thesubsequent section is advantageous for the movement of the specific partin the determination target section.

The first axis determining unit 316 and the second axis determining unit318 respectively determine the advantageous direction along the firstaxis and the advantageous direction along the second axis by referringto the movement vectors calculated by the calculating unit 104. Thus,the first axis determining unit 316 determines the direction indicatedalong the first axis and the second axis determining unit 318 determinesthe direction indicated along the second axis. For example, if the firstaxis serves as the axis in the vertical direction, then the first axisdetermining unit 316 determines whether the upward direction or thedownward direction is indicated along the first axis. Similarly, if thesecond axis serves as the axis in the horizontal direction, then thesecond axis determining unit 318 determines whether the left-handdirection or the right-hand direction is indicated along the secondaxis. In the following explanation, indicating a particular directionalong an axis is referred to as performing “feed action”, in which theuser performs an action of waving the specific part such as a hand so asto indicate the direction of waving.

In order to determine the direction along the second axis in which afeed action is performed, the second axis determining unit 318 canimplement the following determination method. Herein, although theexplanation is given with reference to the second axis determining unit318, the same explanation is also applicable to the first axisdetermining unit 316.

When the velocity of the second axis component of a movement vector (theinner product of the movement vector and the second axis) calculated bythe calculating unit 104 becomes equal to or greater than the athreshold value, the second axis determining unit 318 determines that afeed action is performed in the direction of the movement vector in thesecond axis.

Herein, it is assumed that a position P_(s), which is the start point ofa movement vector R_(s) and which is obtained by the obtaining unit 102at a timing T_(s), has coordinates (x_(s), y_(s)). Similarly, it isassumed that a position P_(s+1), which is the end point of the movementvector R_(s) and which is obtained by the obtaining unit 102 at a timingT_(s+1), has coordinates (x_(s+1), y_(s+1)). In that case, the secondaxis component (the x-axis component) of the movement vector R_(s)becomes x_(s+1)−x_(s), and a second axis component S_(s) of the movementvector R_(s) per unit of time becomesS_(s)=(x_(s+1)−x_(s))/(T_(s+1)−T_(s)).

Moreover, a velocity V_(s) of the second axis component of the movementvector R_(s) is expressed as V_(s)=|S_(s)|. The second axis component ofthe movement vector R_(s) has the positive direction if the sign ofS_(s) is positive or has the negative direction if the sign of S_(s) isnegative. Hence, when the velocity V_(s) becomes equal to or greaterthan a threshold value, the second axis determining unit 318 determinesthat, if the sign of S_(s) is positive, a feed action is performed inthe positive direction along the second axis and determines that, if thesign of S_(s) is negative, a feed action is performed in the negativedirection along the second axis.

Once it is determined that a feed action has been performed, the secondaxis determining unit 318 can stop performing determination until thereis a change in the direction along the second axis component of themovement vector and can make use of the determination result. When thereis a change in the sign of S_(s), the second axis determining unit 318determines that a change has occurred in the direction along the secondaxis component of the movement vector. Meanwhile, the second axisdetermining unit 318 can have zeros sandwiched in between until there isa change in the sign of S_(s). For example, if the sign of S_(s−2) ispositive, the sign of S_(s−1) is 0, and the sign of S_(s) is negative;then, at the point of time of S_(s−1), the second axis determining unit318 does not determine that there is a change in the direction along thesecond axis component. Rather, it is at the point of time of S_(s) thatthe second axis determining unit 318 determines that there is a changein the direction along the second axis component.

Moreover, upon determining that a feed action has been performed, inorder to ensure that that a feed action in the opposite direction(hereinafter, referred to as “return action”) is not determined to havebeen performed, the second axis determining unit 318 can be configuredto ignore a return action performed immediately after the determinationthat a feed action has been performed.

Moreover, alternatively, the second axis determining unit 318 can befirstly configured to not determine that a feed action in the oppositedirection (hereinafter, referred to as “backswing action”) is performedprior to determining that a feed action has been performed. Secondly,upon determining that a feed action has been performed, the second axisdetermining unit 318 can be configured to not determine that a feedaction in the opposite direction (hereinafter, referred to as “returnaction”) is performed. In that case, the second axis determining unit318 refers to the previous section and to the subsequent section of anoperation section that serves as the determination target section, andaccordingly determines whether or not a feed action was performed inthat operation section. Herein, an operation section points to a sectionin which a start point is either set to a second-axis direction turningpoint when such a second-axis direction turning point is present atwhich the direction along the second axis component of the movementvector changes, or set to the initial point indicating theinitially-obtained position when a second-axis direction turning pointis absent; and in which an end point is either set to a second-axisdirection turning point present immediately after the start point, whensuch a second-axis direction turning point is present immediately afterthe start point, or set to the last point that indicates the latestobtained position, when a second-axis direction turning point is notpresent immediately after the start point.

More particularly, firstly, the second axis determining unit 318 setssecond-axis direction turning points for the vectors calculated by thecalculating unit 104. In the third embodiment, it is assumed that thesecond axis determining unit 318 determines in real time whether or nota feed action has been performed in an operation section. However, thatis not the only possible case. Alternatively, that determination canalso be done by performing batch processing, that is, after the gesturehas been completed. Herein, for the sake of convenience, firstly, theexplanation is given for the case when the determination of whether ornot a feed action has been performed in an operation section is done byperforming batch processing. Then, the explanation is given for the casewhen the determination is done in real time.

FIG. 22 is a diagram of an example of setting second-axis directionturning points by the second axis determining unit 318 according to thethird embodiment. In the example illustrated in FIG. 22, “U_(s)”indicates a start point that is the initially-obtained position obtainedby the obtaining unit 102; “U₁” and “U₂” indicate second-axis directionturning points; and “U_(e)” indicates an end point that is the lastposition obtained by the obtaining unit 102. In that case, the secondaxis determining unit 318 sets the section U_(s) to U₁, the section U₁to U₂, and the section U₂ to U_(e) as operation sections in that order;and determines whether or not a feed action has been performed in eachof those operation sections.

More particularly, when the section U_(s) to U₁ is set as the operationsection, the second axis determining unit 318 sets the section U₁ to U₂as the corresponding subsequent section and calculates the evaluationvalues of the operation section and the subsequent section. Accordingly,the second axis determining unit 318 determines whether or not a feedaction was performed in the section U_(s) to U₁ set as the operationsection. In an identical manner, when the section U₁ to U₂ is set as theoperation section, the second axis determining unit 318 sets the sectionU_(s) to U₁ as the corresponding previous section and sets the sectionU₂ to U_(e) as the corresponding subsequent section, and calculates theevaluation values of the operation section, the previous section, andthe subsequent section. Accordingly, the second axis determining unit318 determines whether or not a feed action was performed in the sectionU₁ to U₂ set as the operation section. Similarly, when the section U₂ toU_(e) is set as the operation section, the second axis determining unit318 sets the section U₁ to U₂ as the corresponding previous section andcalculates the evaluation values of the operation section and theprevious section. Accordingly, the second axis determining unit 318determines whether or not a feed action was performed in the section U₂to U_(e) set as the operation section.

Herein, in order to determine that a feed action was performed in anoperation section, the following conditions need to be satisfied: anevaluation value E_(n) of that operation section is equal to or greaterthan a threshold value; and the evaluation value E_(n) of that operationsection is exceeding an evaluation value E_(m) of the correspondingprevious section and greater than an evaluation value E₁ of thecorresponding subsequent section. The evaluation values mentioned aboveare identical to the evaluation values used by the determining unit 312.However, herein, the movement distance can be considered to be the sumof the first axis components of the movement vectors in the operationsection (i.e., modulus of the inner products of the movement vectors andthe first axis). Moreover, the movement velocity can be considered to bethe maximum velocity among the velocities of the first axis componentsof the movement vectors in the operation section or can be considered tobe the average velocity of the velocities of the movement vectors thatare equal to or greater than a predetermined threshold value.Furthermore, the movement acceleration can be considered to be themaximum rate of acceleration from among the rates of acceleration of thefirst axis components of the movement vectors in the operation sectionor can be considered to be the average of the rates of acceleration ofthe movement vectors that are equal to or greater than a predeterminedthreshold value. Meanwhile, when a previous section is not present, theevaluation value E_(m) becomes zero; and when a subsequent section isnot present, the evaluation value E₁ becomes zero.

When the section U_(s) to U₁ is set as the operation section, the secondaxis determining unit 318 does not set a previous section but sets thesection U₁ to U₂ as the corresponds subsequent section. In that case,since the previous section is absent or since the evaluation value E_(n)exceeds the evaluation value E₁, the second axis determining unit 318considers the operation section to be at an advantage. Moreover, if theevaluation value E_(n) is equal to or greater than a threshold value,then the second axis determining unit 318 determines that a feed actionwas performed in the section U_(s) to U₁ set as the operation section.Meanwhile, the second axis determining unit 318 determines that anoperation in the section U₁ to U₂ is a return action in the sectionU_(s) to U₁.

When the section U₁ to U₂ is set as the operation section, the sectionU_(e) to U₁ is set as the corresponding previous section, and thesection U₂ to U_(e) is set as the corresponding subsequent section; theevaluation value E_(n) becomes equal to or smaller than the evaluationvalue E_(m) and the evaluation value E_(n). Hence, the second axisdetermining unit 318 determines that a feed action was not performed inthe section U₁ to U₂ set as the operation section. That is, the secondaxis determining unit 318 determines that an operation in the section U₁to U₂ is a return action in the section U_(s) to U₁.

When the section U₂ to U_(e) is set as the operation section, the secondaxis determining unit 318 sets the section U₁ to U₂ as the previoussection and does not set a subsequent section. In this case, since theevaluation value E_(n) exceeds the evaluation value E_(m) or since thesubsequent section is not present, the second axis determining unit 318considers the operation section to be at an advantage. Moreover, if theevaluation value E_(n) is equal to or greater than a threshold value,the second axis determining unit 318 determines that a feed action wasperformed in the section U₂ to U_(e) set as the operation section.Meanwhile, the second axis determining unit 318 determines that anoperation in the section U₁ to U₂ is a backswing action in the sectionU₂ to U_(e).

Explained above was the case when the determination of whether or not afeed action was performed in an operation section is done by performingbatch processing. In the case of performing such determination in realtime, the task of setting a subsequent section leads to a delay by apredetermined time period in the determination of whether or not a feedaction was performed in the corresponding operation section. Herein,that predetermined time period can be set to, for example, 0.6 seconds.Alternatively, the predetermined time period can be changed depending onthe movement velocity of the specific part in the operation section. Forexample, as the movement velocity goes on increasing, the predeterminedtime period can be reduced. In such a case, during the predeterminedtime period, that is, during the period before the advantageousdirection in the operation section is finalized; the second axisdetermining unit 318 can be configured to perform sequentialdetermination of the advantageous direction in the operation section andto calculate the progress rate until the advantageous direction in theoperation section is finalized. Herein, the progress rate can beexpressed in an identical manner to the progress rate calculated by thedetermining unit 312. Moreover, only when there is a change in thedirection determined to be advantageous in the operation section, thesecond axis determining unit 318 can be configured to output theadvantageous direction (i.e., the determination result). Herein,although the explanation is given with reference to the second axisdetermining unit 318, the same explanation is also applicable to thefirst axis determining unit 316.

FIG. 23 is a diagram illustrating exemplary advantageous directions eachcorresponding to a progress rate and determined by the second axisdetermining unit 318 according to the third embodiment. During thetiming T_(i) to the timing T_(i)+10, the evaluation value of theoperation section falls below a threshold value or becomes equal tosmaller than the corresponding previous section. Hence, thecorresponding progress rate is not calculated and the advantageousdirection in the operation section in also not determined.

During the timing T_(i)+20 to the timing T_(i)+80 (i.e., during theprogress rate of 0 to 0.90), the operation section becomes moreadvantageous as compared to the corresponding previous section, and theadvantageous direction in the operation section is leftward. At thetiming T_(i)+90 (i.e., at the progress rate of 1.00), the operationsection becomes more advantageous as compared to the correspondingprevious section as well as the corresponding subsequent section, andthe advantageous direction in the operation section is finalized to theleft-hand direction. As a result, the second axis determining unit 318determines that a feed action in the left-hand direction was performedin the operation section.

During the timing T_(i)+100 to the timing T_(i)+130, either theoperation section continues without modification even after the feedaction has been performed, or the evaluation value of the operationsection falls below a threshold value or becomes equal to or smallerthan the corresponding previous section. Hence, the correspondingprogress rate is not calculated and the advantageous direction in theoperation section in also not determined.

During the timing T_(i)+140 to the timing T_(i)+150 (i.e., during theprogress rate of 0 to 0.15), the operation section becomes moreadvantageous as compared to the corresponding previous section, and theadvantageous direction in the operation section is rightward.

During the timing T_(i)+160 to the timing T_(i)+220 (i.e., during theprogress rate of 0 to 0.90), the operation section becomes moreadvantageous as compared to the corresponding previous section as wellas the corresponding subsequent section, and the advantageous directionin the operation section is leftward. Moreover, at the timing T_(i)+230(i.e., at the progress rate of 1.00), the operation section becomes moreadvantageous as compared to the corresponding previous section as wellas the corresponding subsequent section, and the advantageous directionin the operation section is finalized to the left-hand direction. As aresult, the second axis determining unit 318 determines that a feedaction in the left-hand direction was performed in the operationsection.

Meanwhile, from the determination results regarding the first axis andthe second axis, the determination result selecting unit 320 selects thedetermination result of the axis that is determined by the determiningunit 312 to be advantageous for the movement of the specific part in adetermination target section.

For example, assume that the advantageous axes determined by thedetermining unit 312 are as illustrated in FIG. 14, assume that thesecond axis determining unit 318 determines (finalizes) that a feedaction in the left-hand direction was performed at the timing T_(i), andassume that the first axis determining unit 316 determines (finalizes)that a feed action in the upward direction was performed at the timingT_(i). In such a case, the determination result selected by thedetermination result selecting unit 320 is as follows.

Regarding the determination, started at the timing T_(i), of theadvantageous axis for the movement of the specific part in adetermination target section, the determining unit 312 finalizes thesecond axis as the advantageous axis at the timing T_(i)+70. Therefore,at the timing T_(i)+70, the determination result selecting unit 320selects the determination result at the timing T_(i) as obtained by thesecond axis determining unit 318 at the timing T_(i), and assumes that afeed action in the left-hand direction along the second axis wasperformed at the timing T_(i).

Meanwhile, during the timing T_(i) to the timing T_(i)+20, thedetermination result selecting unit 320 can be configured to select thedetermination result obtained by the first axis determining unit 316obtained at the timing T_(i). Similarly, during the timing T_(i)+30 tothe timing T_(i)+60, the determination result selecting unit 320 can beconfigured to select the determination result obtained by the secondaxis determining unit 318 obtained at the timing T_(i). That is, duringthe timing T_(i) to the timing T_(i)+20, the feed action candidate atthe timing T_(i) can be set to the feed action in the upward directionalong the first axis; and during the timing T_(i)+30 to the timingT_(i)+60, the feed action candidate at the timing T_(i) can be set tothe feed action in the left-hand direction along the second axis.Meanwhile, the feed action candidate can be a combination of the feedaction and the progress rate. Regarding the direction of a feed actionat a particular timing, it is possible to make use of the direction ofthe feed action in the axis determined by the determining unit 312 to beadvantageous at that timing. Regarding the progress rate, the progressrate obtained by the determining unit 312 can be used.

FIG. 24 is a diagram of feed action candidates each selectedcorresponding to a progress rate by the determination result selectingunit 320. During the timing T_(i) to the timing T_(i)+20 (i.e., duringthe progress rate of 0 to 0.30), the feed action candidate at the timingT_(i) indicates a feed action in the left-hand direction. During thetiming T_(i)+30 to the timing T_(i)+60 (i.e., during the progress rateof 0.45 to 0.90), the feed action candidate at the timing T_(i)indicates a feed action in the upward direction. At the timing T_(i)+70(i.e., at the progress rate of 1.00), the feed action candidate at thetiming T_(i) is finalized to a feed action in the upward direction.

In the case when each of the first axis determining unit 316 and thesecond axis determining unit 318 performs sequential determination ofthe axis advantageous in an operation section while calculating theprogress rates; the determination result selecting unit 320 can beconfigured to select, from the determination results of the first axisand the second axis, the determination result and the progress rate forthe advantageous axis for the movement of the bodily part in adetermination target section determined by the determining unit 312.

Herein, the explanation is given for an example in which, as thedetermination result regarding the axis determined by the determiningunit 312 to be advantageous for the movement of the specific part in adetermination target section, the determination result selecting unit320 selects the determination result regarding the second axis and thecorresponding progress rate. However, the same explanation is alsoapplicable to the case of selecting the determination result regardingthe first axis and the corresponding progress rate.

Once the second axis determining unit 318 starts calculating theprogress rate, the determining unit 312 also starts calculating theprogress rate. That is, once the second axis determining unit 318 startscalculating the progress rate at the timing T_(i), the determining unit312 also starts calculating the progress rate at the timing T_(i). For acertain period of time, the calculation of progress rates is carried on.

Herein, in order to simplify the explanation, during the progress rateof 0.00 to 0.99 calculated by the determining unit 312, if thedetermining unit 312 has determined that the second axis is theadvantageous axis for the movement of the specific part in adetermination target section, it is assumed that the determining unit312 outputs a notification that the second axis is the advantageous axiscandidate. Moreover, at the progress rate of 1.00 calculated by thedetermining unit 312, if the determining unit 312 has determined thatthe second axis is the advantageous axis for the movement of thespecific part in a determination target section, it is assumed that thedetermining unit 312 outputs a notification that the second axis isfinalized as the advantageous axis. In a case other than theabovementioned cases, that is, during the progress rate of 0.00 to 1.00calculated by the determining unit 312, if the determining unit 312 hasdetermined that the first axis is the advantageous axis for the movementof the specific part in a determination target section, it is assumedthat the determining unit 312 does not perform any output.

During a predetermined progress rate (for example, 0.00 to 0.25)calculated by the second axis determining unit 318, once the determiningunit 312 outputs that the second axis is the advantageous axis candidateor that the second axis is finalized as the advantageous axis; thedetermination result selecting unit 320 starts, at that point of time asthe start timing, the selection of the determination result of thesecond axis determining unit 318 and the corresponding progress rate. Inany other case, the determination result selecting unit 320 does notselect the determination result of the second axis determining unit 318and the corresponding progress rate.

Then, if the determining unit 312 stops performing the output during theperiod from the start timing of starting the selection until the timingat which the progress rate calculated by the second axis determiningunit 318 becomes 1.00, then the determination result selecting unit 320stops the selection at the timing at which the determining unit 312 hasstopped performing the output.

Moreover, assume that, during the period from the start timing ofstarting the selection until the timing at which the progress ratecalculated by the second axis determining unit 318 becomes 1.00, thedetermining unit 312 keeps on performing the output and outputs anotification that the second axis is finalized as the advantageous axis.In that case, irrespective of the determination result obtainedsubsequently by the second axis determining unit 318, the determinationresult selecting unit 320 keeps on selecting the progress ratescalculated by the second axis determining unit 318. Then, at the pointof time when the progress rate becomes 1.00, the second axis determiningunit 318 finalizes that a feed action has been performed in thedirection indicated by the determination result of the second axisdetermining unit 318.

Furthermore, assume that, even when the period from the start timing ofstarting the selection reaches the timing at which the progress ratecalculated by the second axis determining unit 318 becomes 1.00, thedetermining unit 312 is still outputting a notification that the secondaxis is the advantageous axis candidate. In that case, until thedetermining unit 312 outputs a notification that the second axis isfinalized as the advantageous axis, the determination result selectingunit 320 keeps on selecting the progress rate of 0.99 as the progressrate calculated by the second axis determining unit 318. Once thedetermining unit 312 outputs a notification that the second axis isfinalized as the advantageous axis, the determination result selectingunit 320 selects the progress rate of 1.00 and finalizes that a feedaction has been performed in the direction indicated by the second axisdetermining unit 318. However, while the determination result selectingunit 320 keeps on selecting the progress rate of 0.99 as the progressrate calculated by the second axis determining unit 318; if thedetermining unit 312 stops performing the output, then the determinationresult selecting unit 320 stops the selection at the timing at which thedetermining unit 312 has stopped performing the output.

Herein, the presenting unit 314 performs presentation according to theselection result obtained by the determination result selecting unit320. In the case when, from the determination results regarding thefirst axis and the second axis, the determination result selecting unit320 selects the determination result of the advantageous axis for themovement of the specific part in a determination target sectiondetermined by the determining unit 312 and selects the correspondingprogress rate; the presenting unit 314 can perform supplementarypresentation, as explained in the second embodiment, until the selectionresult is finalized, that is, while the selection is underway. Once theselection result is finalized, the subsequent presentation is identicalto that explained in the first embodiment.

FIG. 25 is a flowchart illustrating an exemplary sequence of operationsperformed in the recognition apparatus 300 according to the thirdembodiment.

Herein, the operations from Steps S301 to S304 are identical to theoperations from Steps S101 to S104 in the flowchart illustrated in FIG.8.

Subsequently, the section setting unit 310 sets a determination targetsection, in which a start point is either set to the latest principalaxis turning point when principal axis turning points are present or setto the initially-obtained position when principal axis turning pointsare absent, and in which an end point is set to the latest point thatindicates the most-recently obtained position. Moreover, the sectionsetting unit 310 sets a previous section, in which an end point is setto the start point of the determination target section and in which astart point is either set to the direction turning point presentimmediately before the end point when the direction turning points arepresent before the end point or set to the initial point when nodirection turning point is present before the end point. Then, thesection setting unit 310 updates the end point either to the principalaxis turning point present immediately after the start point, when theprincipal axis turning points are present subsequent to the start pointbut before the elapse in a predetermined time period since the settingof the start point, or to the latest point that indicates themost-recently obtained position after the elapse in a predetermined timeperiod, when no principal axis turning point is present subsequent tothe start point but before the elapse in the predetermined time.Furthermore, the section setting unit 310 sets a subsequent section, inwhich a start point is set to the end point of the updated determinationtarget section and in which an end point is either set to the directionturning point present immediately after the start point when thedirection turning points are present subsequent to the start point orset to the latest point present after the elapse in a predetermined timeperiod when latest points are present subsequent to the start point(Step S305).

Then, the determining unit 312 calculates an evaluation value of thedetermination target section, an evaluation value of the correspondingprevious section, and an evaluation value of the correspondingsubsequent section; and, based on those evaluation values, determineswhich of the first axis and the second axis is advantageous for themovement of the specific part in the determination target section (StepS306).

The first axis determining unit 316 refers to the movement vectorscalculated by the calculating unit 104 and accordingly determines theadvantageous direction along the first axis (Step S307).

Similarly, the second axis determining unit 318 refers to the movementvectors calculated by the calculating unit 104 and accordinglydetermines the advantageous direction along the second axis (Step S308).

Then, from the determination results regarding the first axis and thesecond axis, the determination result selecting unit 320 selects thedetermination result of the axis that is determined by the determiningunit 312 to be advantageous for the movement of the specific part in adetermination target section (Step S310).

Every time the obtaining unit 102 obtains the latest position of thespecific part of the photographic subject (Step S301), the operationsfrom Step S302 to Step S310 are repeated, and the presenting unit 314performs presentation according to the determination result of thedetermination result selecting unit 320.

Explained below is an exemplary sequence of operations performed by thedetermining unit 312. The sequence of operations performed by thedetermining unit 312 is a combination of the flowchart illustrated inFIG. 10 and the flowchart illustrated in FIG. 20. More particularly, inthe flowchart illustrated in FIG. 10, when the evaluation value E_(c) isnot equal to or smaller than the evaluation value E_(b) (No at StepS131), the system control proceeds not to Step S133 but to Step S241illustrated in the flowchart in FIG. 20.

FIG. 26 is a flowchart for explaining an exemplary sequence ofoperations performed by the second axis determining unit 318 in therecognition apparatus 300 according to the third embodiment. Herein,although not explained, the same sequence of operations is alsoperformed by the first axis determining unit 316. Moreover, theexplanation with reference to FIG. 26 is given for a case when thedetermination of whether or not a feed action was performed in anoperation section is done in real time.

Firstly, as a determination target section, the second axis determiningunit 318 sets an operation section in which a start point is either setto a second-axis direction turning point when such a second-axisdirection turning point is present at which the direction along thesecond axis component of the movement vector changes, or set to theinitial point indicating the initially-obtained position when asecond-axis direction turning point is absent; and in which an end pointis set to the latest point that indicates the most-recently obtainedposition. Then, the second axis determining unit 318 calculates anevaluation value of the operation section and checks whether or not theevaluation is equal to or greater than a threshold value (Step S351).

If the evaluation value of the operation section is equal to or greaterthan a threshold value (Yes at Step S351), then the second axisdetermining unit 318 checks whether or not the operation section is moreadvantageous as compared to the previous condition (Step S352). Moreparticularly, the second axis determining unit 318 sets a previoussection in which an end point is set to the start point of the operationsection and in which a start point is either set to a second-axisdirection turning point present immediately before the end point, whenthe second-axis direction turning points are present before the endpoint, or set to the initial point when a second-axis direction turningpoint is not present before the end point. Then, the second axisdetermining unit 318 calculates an evaluation value of the previoussection and checks whether or not the evaluation value of the operationsection is exceeding the evaluation value of the previous section.

When the operation section is more advantageous as compared to thecorresponding previous condition (Yes at Step S352), that is, when theevaluation value of the operation section is exceeding the evaluationvalue of the corresponding previous section; the second axis determiningunit 318 checks whether the operation section is more advantageous ascompared to the subsequent condition (Step S353). More particularly, thesecond axis determining unit 318 updates the end point either to thesecond-axis direction turning point present immediately after the startpoint of the operation section, when such a second-axis directionturning point is present immediately after the start point of theoperation section before the elapse in a predetermined time period sincethe setting of the operation section, or to the latest point thatindicates the most-recently obtained position after the elapse in apredetermined time period, when no second-axis direction turning pointis present immediately after the start point of the operation sectionbefore the elapse in the predetermined time. Moreover, the second axisdetermining unit 318 sets a subsequent section, in which a start pointis set to the end point of the updated operation section and in which anend point is either set to the second-axis direction turning pointsubsequent to the start point when such a second-axis direction turningpoint is present subsequent to the start point or set to the latestpoint present after the elapse in a predetermined time period whenlatest points are present subsequent to the start point. Then, thesecond axis determining unit 318 calculates an evaluation value of thesubsequent section and checks whether or not the evaluation value of theoperation section is exceeding the evaluation value of the subsequentsection.

When the operation section is more advantageous as compared to thecorresponding subsequent condition (Yes at Step S353), that is, when theevaluation value of the operation section is exceeding the evaluationvalue of the corresponding subsequent section; the second axisdetermining unit 318 determines that a feed action in the direction ofthe movement vectors in the second axis was performed in the operationsection (Step S354).

On the other hand, when the evaluation value of the operation section issmaller than the threshold value (No at Step S351) or when the operationsection is not advantageous as compared to the previous condition (No atStep S352) or when the operation section is not advantageous as comparedto the subsequent condition (No at Step S353); the second axisdetermining unit 318 determines that the action performed in theoperation section was not a feed action (Step S355).

As described above, in the third embodiment, the gesture in adetermination target section is recognized using the correspondingprevious section and the corresponding subsequent section. Thus,according to the third embodiment, it becomes possible to determinewhether or not the gesture in a determination target section is noise.Because of that, even if a movement representing noise is included in agesture, such movement can be recognized correctly thereby enablingprevention of a decline in the recognition accuracy. For example, evenif a movement representing noise is included at the start and at the endof a gesture, such movement can be recognized correctly. Moreover,according to the third embodiment, it is also possible to recognize thedirection indicated along each axis.

Fourth Embodiment

In the first to third embodiments described above, the explanation isgiven for a case in which two axes are included in the coordinate systemrepresenting the positions of a specific part of a photographic subject.That is, there are two axes that are considered to be the determinationtargets for the gestures performed by the specific part. In a fourthembodiment, the explanation is given for a case in which three or moreaxes are included in the coordinate system representing the positions ofa specific part of a photographic subject. Herein, it is assumed thatthe axes intersect with each other at a single point.

FIG. 27 is a diagram illustrating an exemplary coordinate systemaccording to the fourth embodiment. In the coordinate system illustratedin the example in FIG. 27, a first axis to a third axis are setcorresponding to mutually independent directions of x, y, and z,respectively that define a three-dimensional space. Hence, determinationof gestures can be performed in the up-down direction, the left-rightdirection, and the front-back direction.

FIG. 28 is a diagram illustrating another exemplary coordinate systemaccording to the fourth embodiment. In the coordinate system illustratedin FIG. 28, it is possible to set the axes corresponding to threedifferent directions on the same plane (for example, three axes rotatedby 120° from each other on the same plane).

The principal axis selecting unit 106 calculates, for each movementvector that is calculated, a first inner product that is the innerproduct of the movement vector and the first axis, a second innerproduct that is the inner product of the movement vector and the secondaxis, and a third inner product that is the inner product of themovement vector and the third axis. Then, of the first to third axes,the principal axis selecting unit 106 selects the axis that is used incalculating the inner product having the greatest modulus, from amongthe first inner product to the third inner product, as the principalaxis.

Consider the case when more than one inner product has the greatestmodulus. In that case, if the axes that are used in calculating theinner products having the greatest modulus include the previousprincipal axis, then the principal axis selecting unit 106 selects theprevious principal axis as the current principal axis. On the otherhand, if the axes that are used in calculating the inner products havingthe greatest modulus do not include the previous principal axis, thenthe principal axis selecting unit 106 selects, as the principal axis, anarbitrary axis from among the axes that are used in calculating theinner products having the greatest modulus.

Application

Explained below are exemplary applications of the recognition apparatusaccording to the embodiments described above. The applications explainedbelow are only exemplary, and the recognition apparatus can also beimplemented in other devices.

First Application

FIG. 29 is a diagram illustrating a television receiver 400 according toa first application. In the television receiver 400 illustrated in FIG.29, the recognition apparatus according to any one of the embodimentsdescribed above is implemented, and the channel operations are performedvia gestures.

The television receiver 400 has a camera 402 installed therein forcapturing images of the viewer. As the specific part, the televisionreceiver 400 detects and follows a hand 404 of the viewer in order torecognize feed actions. It is illustrated that a television screen 406is currently displaying a channel 4. When the viewer performs a feedaction of swiftly moving the hand 404 leftward, the channel can bechanged to a channel 3. Similarly, when the viewer performs a feedaction of swiftly moving the hand 404 rightward, the channel can bechanged to a channel 5.

Second Application

FIG. 30 is a diagram illustrating a personal computer 500 according to asecond application. In the personal computer 500 illustrated in FIG. 30,the recognition apparatus according to any one of the embodimentsdescribed above is implemented. Herein, the web contents or the textdisplayed in a window 508 on a screen 506 can be scrolled via gesturesof a hand 504.

The personal computer 500 has a camera 502 installed therein forcapturing images of the user. As the specific part, the personalcomputer 500 detects and follows the hand 504 of the user in order torecognize feed actions. Then, in the direction of the feed action thathas been detected, the personal computer 500 issues a scroll command forscrolling in that direction. At that time, the amount of scrolling,which is displayed on a scroll bar 510, can be set to a predeterminedconstant amount. Alternatively, the amount of scrolling can be changeddepending on the speed of the feed action. For example, faster the feedaction, greater can be the amount of scrolling.

As a result, the user can scroll the contents, which are displayed inthe window 508, in the direction of the feed action and thus browse theweb contents or the text.

Third Embodiment

FIG. 31 is a diagram illustrating a medical information terminal 600according to a third application. In the medical information terminal600 illustrated in FIG. 31, the recognition apparatus according to anyone of the embodiments described above is implemented. The medicalinformation terminal 600 is used in providing information to a doctorwho is performing a surgery.

Since the doctor who is performing a surgery has to keep a hand 604clean, it is not possible to use the hand 604 to directly operate themedical information terminal 600, which provides information such as themonitoring information or the laboratory results of the patient.

For that reason, by implementing the recognition apparatus according toany one of the embodiments described above in the medical informationterminal 600, it becomes possible for the doctor to operate the medicalinformation terminal 600 in a contactless manner via gestures of thehand 604.

The medical information terminal 600 has a camera 602 installed thereinfor capturing images of the doctor. As the specific part, the medicalinformation terminal 600 detects and follows the hand 604 of the doctorin order to recognize feed actions. Then, in the direction of the feedaction that has been detected, the medical information terminal 600issues a scroll command for scrolling in that direction or issues achange command for changing the display information.

MODIFICATION EXAMPLE

The invention is not limited to the embodiments described above and itis possible to implement various modifications within the scope of theinvention.

For example, the first embodiment and the second embodiment can becombined together. In that case, the section setting unit, thedetermining unit, and the presenting unit perform operations identicalto the operations described in the third embodiment. Moreover, in thethird embodiment, although the section setting unit 310 sets previoussections as well as subsequent sections with respect to determinationtarget sections, the section setting unit 310 can alternatively beconfigured to set either previous sections or subsequent sections asdescribed in the first embodiment or in the second embodiment.

As described above, the television receiver 400, the personal computer500, and the medical information terminal 600 are explained asapplications of the recognition apparatus. Besides, the recognitionapparatus can also be implemented in other devices such as gameconsoles. That is, the recognition apparatus can also be implemented indevices that enable the user to perform feed actions regarding devicecontrol menus or feed actions regarding displayed contents with the useof gesture interfaces provided for easy device operations without havingto hold or wear remote controllers, sensors, markers, or the like.

Meanwhile, the recognition apparatus according to each embodiment andeach modification example described above has the hardware configurationof a commonly-used computer that includes a control device such as acentral processing unit (CPU), a memory device such as a read onlymemory (ROM) or a random access memory (RAM), an external memory devicesuch as a hard disk drive (HDD) or a solid state drive (SSD), a displaydevice such as a display, an input device such as a mouse or a keyboard,and a communication device such as a communication interface (I/F).

Meanwhile, a recognition program, which is executed in the recognitionapparatus according to each embodiment and each modification exampledescribed above, can be provided in the form of an installable orexecutable file on a computer-readable recording medium such as acompact disk read only memory (CD-ROM), a compact disk recordable(CD-R), a digital versatile disk (DVD), or a flexible disk (FD).

Alternatively, the recognition program, which is executed in therecognition apparatus according to each embodiment and each modificationexample described above, can be saved on a computer connected to anetwork and downloaded from that computer via the network. Stillalternatively, the recognition program, which is executed in therecognition apparatus according to each embodiment and each modificationexample described above, can be made available for distribution throughthe network such as the Internet. Still alternatively, the recognitionprogram, which is executed in the recognition apparatus according toeach embodiment and each modification example described above, can bestored in advance in a ROM or the like.

Herein, the recognition program, which is executed in the recognitionapparatus according to each embodiment and each modification exampledescribed above, contains modules for implementing the functions of theabovementioned constituent elements in a computer. Regarding the actualhardware, a CPU retrieves the recognition program from, for example, anHDD and runs it so that the recognition program is loaded in a RAM. As aresult, the functions of the abovementioned constituent elements areimplemented in the RAM.

In this way, according to an aspect of the invention, it becomespossible to prevent a decline in the recognition accuracy.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

1.-20. (canceled)
 21. A recognition apparatus comprising: an obtainingunit configured to obtain, in chronological order, positions of aspecific part of a photographic subject that are represented in acoordinate system having a first axis to an n-th axis (n≧2); acalculating unit configured to calculate a movement vector of thespecific part at each of the positions obtained in chronological order;a principal axis selecting unit configured to calculate, for each ofmovement vectors that has been calculated, a first inner product that isan inner product of the movement vector and the first axis to an n-thinner product that is an inner product of the movement vector and then-th axis, and select, as a principal axis, an axis that is used incalculating the inner product, from among the first inner product to then-th inner product, having the greatest modulus; a turning point settingunit configured to set a position at which there is a change in theprincipal axis as a principal axis turning point and a direction turningpoint, and set a position at which there is a change in the direction ofany of the movement vectors in the principal axis component as thedirection turning point; a section setting unit configured to set adetermination target section, in which a start point is either set tothe latest principal axis turning point when principal axis turningpoints are present or set to an initial point indicating theinitially-obtained position when the principal axis turning points areabsent, and in which an end point is set to a latest point thatindicates the most-recently obtained position, set a previous section,in which an end point is set to the start point of the determinationtarget section and in which a start point is either set to a directionturning point present immediately before the end point when thedirection turning points are present before the end point or set to theinitial point when the direction turning points are not presentimmediately before the end point; a determining unit configured tocalculate an evaluation value of the determination target section and anevaluation value of the immediately previous section and, based on theevaluation value of the determination target section and the evaluationvalue of the previous section, determine which of the first axis to then-th axis is advantageous for movement of the specific part in thedetermination target section; and a presenting unit configured toperform presentation according to a determination result obtained by thedetermining unit, wherein the section setting unit updates the end pointof the determination target section either to the principal axis turningpoint that is present immediately after the start point of thedetermination target section when the principal axis turning points arepresent subsequent to the start point but before the elapse of apredetermined time period since the setting of the start point, or to alatest point that is present after the elapse of a predetermined timeperiod, when no principal axis turning point is present subsequent tothe start point but before the elapse of the predetermined time period,and sets a subsequent section, in which a start point is set to the endpoint of an updated determination target section and in which an endpoint is either set to the direction turning point that is presentimmediately after the start point when the direction turning points arepresent subsequent to the start point or set to a latest point that ispresent after the elapse of the predetermined time period when latestpoints are present subsequent to the start point, and the determiningunit further calculates an evaluation value of the subsequent sectionand further determines which of the first axis to the n-th axis isadvantageous for movement of the specific part in the determinationtarget section based on the evaluation value of the subsequent section.22. The apparatus according to claim 21, wherein when the evaluationvalue of the determination target section is exceeding the evaluationvalue of the previous section or when the previous section is notpresent, when the evaluation value of the determination target sectionis exceeding the evaluation value of the subsequent section or when thesubsequent section is not present, the determining unit determines thatthe principal axis in the determination target section is advantageousfor movement of the specific part in the determination target section,when the evaluation value of the determination target section is equalto or smaller than the evaluation value of the previous section, thedetermining unit determines that the principal axis in the previoussection is advantageous for movement of the specific part in thedetermination target section, and when the evaluation value of thedetermination target section is equal to or smaller than the evaluationvalue of the subsequent section, the determining unit determines thatthe principal axis in the subsequent section is advantageous formovement of the specific part in the determination target section. 23.The apparatus according to claim 21, wherein the determining unitperforms sequential determination, within the predetermined time period,of an axis advantageous for movement of the specific part in thedetermination target section, and the presenting unit further performssupplementary presentation within the predetermined time periodaccording to a determination result obtained by the determining unit.24. The apparatus according to claim 21, further comprising: a firstaxis determining unit to an n-th axis determining unit configured torespectively determine an advantageous direction along the first axis tothe n-th axis by referring to the movement vectors that are calculated;and a determination result selecting unit configured to select, fromdetermination results regarding the first axis to the n-th axis, adetermination result regarding an axis that is determined by thedetermining unit to be advantageous for movement of the specific part inthe determination target section, wherein the presenting unit performspresentation according to a selection result obtained by thedetermination result selecting unit.
 25. The apparatus according toclaim 24, wherein each of the first axis determining unit to the n-thaxis determining unit further calculates a progress rate ofcorresponding determination result regarding the first axis to the n-thaxis, when the progress rate of a determination result of any one axisfrom among the first axis to the n-th axis is within a predeterminedprogress rate and when the any one axis is determined by the determiningunit to be advantageous for movement of the specific part in thedetermination target section, the determination result selecting unitkeeps on selecting the determination result and the progress rate untilthe progress rate represents a progress rate that finalizes thedetermination result and until the any one axis is finalized as an axisdetermined by the determining unit to be advantageous for movement ofthe specific part in the determination target section, and while thedetermination result selecting unit keeps on selecting the determinationresult, the presenting unit further performs supplementary presentationaccording to the selection result.
 26. A recognition apparatuscomprising: an obtaining unit configured to obtain, in chronologicalorder, positions of a specific part of a photographic subject that arerepresented in a coordinate system having a first axis to an n-th axis(n≦2); a calculating unit configured to calculate a movement vector ofthe specific part at each of the positions obtained in chronologicalorder; a principal axis selecting unit configured to calculate, for eachof the movement vectors that has been calculated, a first inner productthat is the inner product of the movement vector and the first axis toan n-th inner product that is the inner product of the movement vectorand the n-th axis, and select, as a principal axis, an axis that is usedin calculating the inner product, from among the first inner product tothe n-th inner product, having the greatest modulus; a turning pointsetting unit configured to set a position at which there is a change inthe principal axis as a principal axis turning point and a directionturning point, and set a position at which there is a change in thedirection of any of the movement vectors in the principal axis componentas the direction turning point; a section setting unit configured to seta determination target section, in which a start point is either set tothe latest principal axis turning point when principal axis turningpoints are present or set to an initial point indicating theinitially-obtained position when the principal axis turning points areabsent, and in which an end point is set to the principal axis turningpoint present immediately after the start point of the determinationtarget section, when the principal axis turning points are presentsubsequent to the start point but before the elapse of a predeterminedtime period since the setting of the start point, or set to a latestpoint present after the elapse of a predetermined time period, when noprincipal axis turning point is present subsequent to the start pointbut before the elapse of the predetermined time period, and set asubsequent section, in which a start point is set to the end point ofthe determination target section and in which an end point is either setto the direction turning point present immediately after the start pointwhen the direction turning points are present subsequent to the startpoint or set to the latest point when the latest point is presentsubsequent to the start point; a determining unit configured tocalculate an evaluation value of the determination target section and anevaluation value of the subsequent section and, based on the evaluationvalue of the determination target section and the evaluation value ofthe subsequent section, determine which of the first axis to the n-thaxis is advantageous for movement of the specific part in thedetermination target section; a first axis determining unit to an n-thaxis determining unit configured to respectively determine anadvantageous direction along the first axis to the n-th axis byreferring to movement vectors that are calculated; a determinationresult selecting unit configured to select, from determination resultsregarding the first axis to the n-th axis, a determination resultregarding an axis that is determined by the determining unit to beadvantageous for movement of the specific part in the determinationtarget section, and a presenting unit configured to perform presentationaccording to a selection result obtained by the determination resultselecting unit.
 27. The apparatus according to claim 26, wherein each ofthe first axis determining unit to the n-th axis determining unitfurther calculates a progress rate of corresponding determination resultregarding the first axis to the n-th axis, when the progress rate of adetermination result of any one axis from among the first axis to then-th axis is within a predetermined progress rate and when the any oneaxis is determined by the determining unit to be advantageous formovement of the specific part in the determination target section, thedetermination result selecting unit keeps on selecting the determinationresult and the progress rate until the progress rate represents aprogress rate that finalizes the determination result and until the anyone axis is finalized as an axis determined by the determining unit tobe advantageous for movement of the specific part in the determinationtarget section, and while the determination result selecting unit keepson selecting the determination result, the presenting unit furtherperforms supplementary presentation according to the selection result.